[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:05 dist-test-slave-7q7m krb5kdc[10770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:05 dist-test-slave-7q7m krb5kdc[10770](info): set up 2 sockets
Sep 24 07:14:05 dist-test-slave-7q7m krb5kdc[10770](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260924 07:14:05.916044 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:05 dist-test-slave-7q7m krb5kdc[10770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:14:06.024452 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:41829
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:41829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:06.101275 10783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:06.101480 10783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:06.101511 10783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:06.102617 10783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:06.102655 10783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:06.102674 10783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:06.102689 10783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:06.102705 10783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:06.104128 10783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:41829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:41829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:06.104350 10783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:06.104557 10783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 07:14:06.110105 10783 server_base.cc:1034] running on GCE node
W20260924 07:14:06.110080 10788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:06.110100 10789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:06.110085 10791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:06.110531 10783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:06.110809 10783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:06.112032 10783 hybrid_clock.cc:648] HybridClock initialized: now 1790234046112004 us; error 39 us; skew 500 ppm
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[10770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234046, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:06.114470 10783 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:14:06.114877 10783 webserver.cc:460] Webserver started at http://127.10.130.254:43691/ using document root <none> and password file <none>
I20260924 07:14:06.115089 10783 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:06.115136 10783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:06.115258 10783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:06.116361 10783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "ea1b2146496f4fd28407919f70437f34"
format_stamp: "Formatted at 2026-09-24 07:14:06 on dist-test-slave-7q7m"
server_key: "6c8c6d0415c728ad0e0ee7cd80f81038"
server_key_iv: "adeb8e653631dc76aa6befd4438d171c"
server_key_version: "encryptionkey@0"
I20260924 07:14:06.116701 10783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "ea1b2146496f4fd28407919f70437f34"
format_stamp: "Formatted at 2026-09-24 07:14:06 on dist-test-slave-7q7m"
server_key: "6c8c6d0415c728ad0e0ee7cd80f81038"
server_key_iv: "adeb8e653631dc76aa6befd4438d171c"
server_key_version: "encryptionkey@0"
I20260924 07:14:06.117944 10783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:06.118589 10798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:06.118746 10783 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:06.118827 10783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "ea1b2146496f4fd28407919f70437f34"
format_stamp: "Formatted at 2026-09-24 07:14:06 on dist-test-slave-7q7m"
server_key: "6c8c6d0415c728ad0e0ee7cd80f81038"
server_key_iv: "adeb8e653631dc76aa6befd4438d171c"
server_key_version: "encryptionkey@0"
I20260924 07:14:06.118908 10783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:06.127751 10783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:06.129850 10783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:06.129992 10783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:06.133785 10783 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:41829
I20260924 07:14:06.133846 10850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:41829 every 8 connection(s)
I20260924 07:14:06.134156 10783 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:06.134989 10851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:06.137245 10851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34: Bootstrap starting.
I20260924 07:14:06.137792 10851 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:06.138047 10851 log.cc:824] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:06.138406 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 10783
I20260924 07:14:06.138494 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:06.138796 10851 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34: No bootstrap required, opened a new log
I20260924 07:14:06.139101 10763 external_mini_cluster.cc:1428] Setting key 46a6472e3fed02872424cde7aad23a12
I20260924 07:14:06.140496 10851 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea1b2146496f4fd28407919f70437f34" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 41829 } }
I20260924 07:14:06.140630 10851 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:06.140650 10851 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea1b2146496f4fd28407919f70437f34, State: Initialized, Role: FOLLOWER
I20260924 07:14:06.140758 10851 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea1b2146496f4fd28407919f70437f34" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 41829 } }
I20260924 07:14:06.140829 10851 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:06.140851 10851 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:06.140898 10851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:06.141703 10851 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea1b2146496f4fd28407919f70437f34" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 41829 } }
I20260924 07:14:06.141790 10851 leader_election.cc:302] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea1b2146496f4fd28407919f70437f34; no voters:
I20260924 07:14:06.141942 10851 leader_election.cc:288] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:06.141979 10856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:06.142118 10856 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [term 1 LEADER]: Becoming Leader. State: Replica: ea1b2146496f4fd28407919f70437f34, State: Running, Role: LEADER
I20260924 07:14:06.142233 10851 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:06.142243 10856 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea1b2146496f4fd28407919f70437f34" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 41829 } }
I20260924 07:14:06.142640 10858 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea1b2146496f4fd28407919f70437f34. Latest consensus state: current_term: 1 leader_uuid: "ea1b2146496f4fd28407919f70437f34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea1b2146496f4fd28407919f70437f34" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 41829 } } }
I20260924 07:14:06.142717 10858 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:06.142925 10857 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea1b2146496f4fd28407919f70437f34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea1b2146496f4fd28407919f70437f34" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 41829 } } }
I20260924 07:14:06.143195 10862 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:06.143292 10857 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:06.143826 10862 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:06.145323 10862 catalog_manager.cc:1347] Generated new cluster ID: 7ebcccf109d84a4494b9e20a260e6167
I20260924 07:14:06.145370 10862 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[10770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234045, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:06.155786 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:06.139922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50584 (local address 127.10.130.254:41829)
0924 07:14:06.140218 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:06.140227 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:06.140270 (+ 43us) server_negotiation.cc:407] Connection header received
0924 07:14:06.142051 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:06.142057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:06.142114 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:06.143655 (+ 1541us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:06.144324 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.145644 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.147052 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.147716 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.147836 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:06.151898 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:06.151902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:06.151902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:06.151914 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:06.153748 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:06.154222 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:06.154224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:06.154225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:06.154318 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:06.154588 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:06.154588 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:06.154588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:06.154854 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:06.154966 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:06.155267 (+ 301us) server_negotiation.cc:299] Negotiation successful
0924 07:14:06.155393 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":90,"threads_started":1}
I20260924 07:14:06.158933 10862 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:06.159399 10862 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:06.164330 10862 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea1b2146496f4fd28407919f70437f34: Generated new TSK 0
I20260924 07:14:06.164496 10862 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:14:06.191942 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:41829
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:06.269042 10877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:06.269248 10877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:06.269270 10877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:06.270423 10877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:06.270468 10877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:06.270493 10877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:06.271934 10877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:41829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:06.272167 10877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:06.272387 10877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:06.277001 10882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:06.277009 10885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:06.277189 10877 server_base.cc:1034] running on GCE node
W20260924 07:14:06.277222 10883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:06.277411 10877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:06.277608 10877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:06.278755 10877 hybrid_clock.cc:648] HybridClock initialized: now 1790234046278744 us; error 28 us; skew 500 ppm
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[10770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234046, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:06.281100 10877 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:14:06.281530 10877 webserver.cc:460] Webserver started at http://127.10.130.193:46071/ using document root <none> and password file <none>
I20260924 07:14:06.281747 10877 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:06.281795 10877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:06.281919 10877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:06.282944 10877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "aad28101e4ce4fc78f3cd769e28856ec"
format_stamp: "Formatted at 2026-09-24 07:14:06 on dist-test-slave-7q7m"
server_key: "add50b9ccce72edb2a6508abd7dfa288"
server_key_iv: "2a0d2bce8a17a04f3c9a63a310131412"
server_key_version: "encryptionkey@0"
I20260924 07:14:06.283265 10877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "aad28101e4ce4fc78f3cd769e28856ec"
format_stamp: "Formatted at 2026-09-24 07:14:06 on dist-test-slave-7q7m"
server_key: "add50b9ccce72edb2a6508abd7dfa288"
server_key_iv: "2a0d2bce8a17a04f3c9a63a310131412"
server_key_version: "encryptionkey@0"
I20260924 07:14:06.284536 10877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:14:06.285156 10892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:06.285359 10877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:06.285431 10877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "aad28101e4ce4fc78f3cd769e28856ec"
format_stamp: "Formatted at 2026-09-24 07:14:06 on dist-test-slave-7q7m"
server_key: "add50b9ccce72edb2a6508abd7dfa288"
server_key_iv: "2a0d2bce8a17a04f3c9a63a310131412"
server_key_version: "encryptionkey@0"
I20260924 07:14:06.285490 10877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:06.292296 10877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:06.293984 10877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:06.294107 10877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:06.294303 10877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:06.294569 10877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:06.294600 10877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:06.294626 10877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:06.294642 10877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:06.299938 10877 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:38717
I20260924 07:14:06.300020 11005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:38717 every 8 connection(s)
I20260924 07:14:06.300282 10877 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:14:06.305637 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 10877
I20260924 07:14:06.305727 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[10770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234046, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:06.305871 10763 external_mini_cluster.cc:1428] Setting key 87ff21b6e6cd04f1004f2281fdf588a2
I20260924 07:14:06.309571 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:06.300918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:35311 (local address 127.10.130.254:41829)
0924 07:14:06.301001 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:06.301004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:06.301121 (+ 117us) server_negotiation.cc:407] Connection header received
0924 07:14:06.301571 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:06.301572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:06.301582 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:06.301630 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:06.302503 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.303000 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.303929 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.304189 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.304239 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:06.306320 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:06.306321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:06.306322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:06.306329 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:06.308020 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:06.308481 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:06.308482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:06.308482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:06.308565 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:06.308933 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:06.308934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:06.308935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:06.309051 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:06.309108 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:06.309396 (+ 288us) server_negotiation.cc:299] Negotiation successful
0924 07:14:06.309480 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260924 07:14:06.309653 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:06.300880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:41829 (local address 127.10.130.193:35311)
0924 07:14:06.301012 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:06.301046 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:06.301494 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:06.301706 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:06.301709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:06.302236 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:06.302435 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:06.302440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.303061 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.303061 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:06.303863 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:06.303865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.304263 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.304264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:06.304372 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:06.304795 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:06.304802 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:06.306162 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:06.308119 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:06.308126 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:06.308128 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:06.308402 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:06.308646 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:06.308647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:06.308647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:06.308840 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:06.309111 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:06.309117 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:06.309275 (+ 158us) client_negotiation.cc:769] Sending connection context
0924 07:14:06.309364 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 07:14:06.309483 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
I20260924 07:14:06.310077 11006 heartbeater.cc:347] Connected to a master server at 127.10.130.254:41829
I20260924 07:14:06.310158 11006 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:06.310317 11006 heartbeater.cc:511] Master 127.10.130.254:41829 requested a full tablet report, sending...
I20260924 07:14:06.310851 10815 ts_manager.cc:194] Registered new tserver with Master: aad28101e4ce4fc78f3cd769e28856ec (127.10.130.193:38717)
I20260924 07:14:06.311738 10815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:35311
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:14:06.330751 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:41829
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:06.409556 11011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:06.409765 11011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:06.409788 11011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:06.410887 11011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:06.410923 11011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:06.410949 11011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:06.412357 11011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:41829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:06.412551 11011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:06.412722 11011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:06.417424 11019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:06.417431 11016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:06.417444 11017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:06.417661 11011 server_base.cc:1034] running on GCE node
I20260924 07:14:06.417941 11011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:06.418164 11011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:06.419297 11011 hybrid_clock.cc:648] HybridClock initialized: now 1790234046419283 us; error 29 us; skew 500 ppm
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[10770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234046, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:06.421967 11011 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:14:06.422418 11011 webserver.cc:460] Webserver started at http://127.10.130.194:43611/ using document root <none> and password file <none>
I20260924 07:14:06.422616 11011 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:06.422703 11011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:06.422838 11011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:06.424198 11011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "573c2506a03a45ee992fef9a4a821b74"
format_stamp: "Formatted at 2026-09-24 07:14:06 on dist-test-slave-7q7m"
server_key: "806d665443ac6ae2322ce8f03c588321"
server_key_iv: "dd992afd25ddb9d7f5713adf9460aca5"
server_key_version: "encryptionkey@0"
I20260924 07:14:06.424640 11011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "573c2506a03a45ee992fef9a4a821b74"
format_stamp: "Formatted at 2026-09-24 07:14:06 on dist-test-slave-7q7m"
server_key: "806d665443ac6ae2322ce8f03c588321"
server_key_iv: "dd992afd25ddb9d7f5713adf9460aca5"
server_key_version: "encryptionkey@0"
I20260924 07:14:06.426105 11011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:06.426975 11026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:06.427234 11011 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:06.427325 11011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "573c2506a03a45ee992fef9a4a821b74"
format_stamp: "Formatted at 2026-09-24 07:14:06 on dist-test-slave-7q7m"
server_key: "806d665443ac6ae2322ce8f03c588321"
server_key_iv: "dd992afd25ddb9d7f5713adf9460aca5"
server_key_version: "encryptionkey@0"
I20260924 07:14:06.427451 11011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:06.442307 11011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:06.444085 11011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:06.444188 11011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:06.444362 11011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:06.444607 11011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:06.444634 11011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:06.444661 11011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:06.444674 11011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:06.450062 11011 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:42863
I20260924 07:14:06.450129 11139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:42863 every 8 connection(s)
I20260924 07:14:06.450393 11011 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:14:06.454945 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 11011
I20260924 07:14:06.455024 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:06.455173 10763 external_mini_cluster.cc:1428] Setting key aa474c7e698640c81806c2da1672a90b
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[10770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234046, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:06.459022 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:06.451092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:45075 (local address 127.10.130.254:41829)
0924 07:14:06.451147 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:06.451149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:06.451321 (+ 172us) server_negotiation.cc:407] Connection header received
0924 07:14:06.451825 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:06.451826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:06.451836 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:06.451899 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:06.452819 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.453321 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.454236 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.454421 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.454468 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:06.456519 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:06.456521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:06.456521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:06.456529 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:06.457754 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:06.458152 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:06.458154 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:06.458154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:06.458215 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:06.458425 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:06.458426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:06.458426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:06.458546 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:06.458608 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:06.458827 (+ 219us) server_negotiation.cc:299] Negotiation successful
0924 07:14:06.458926 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 07:14:06.459126 11142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:06.451056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:41829 (local address 127.10.130.194:45075)
0924 07:14:06.451205 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:06.451244 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:06.451714 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:06.451965 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:06.451968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:06.452536 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:06.452732 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:06.452736 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.453390 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.453390 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:06.454166 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:06.454168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.454481 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.454482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:06.454585 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:06.455050 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:06.455060 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:06.456432 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:06.457834 (+ 1402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:06.457840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:06.457842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:06.458080 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:06.458275 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:06.458275 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:06.458276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:06.458366 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:06.458599 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:06.458602 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:06.458709 (+ 107us) client_negotiation.cc:769] Sending connection context
0924 07:14:06.458784 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 07:14:06.458913 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260924 07:14:06.459561 11140 heartbeater.cc:347] Connected to a master server at 127.10.130.254:41829
I20260924 07:14:06.459647 11140 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:06.459813 11140 heartbeater.cc:511] Master 127.10.130.254:41829 requested a full tablet report, sending...
I20260924 07:14:06.460186 10815 ts_manager.cc:194] Registered new tserver with Master: 573c2506a03a45ee992fef9a4a821b74 (127.10.130.194:42863)
I20260924 07:14:06.460548 10815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:45075
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:14:06.478276 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:41829
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:06.556118 11145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:06.556309 11145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:06.556339 11145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:06.557466 11145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:06.557557 11145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:06.557610 11145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:06.559320 11145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:41829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:06.559548 11145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:06.559762 11145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:06.564391 11151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:06.564404 11150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:06.564527 11153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:06.564425 11145 server_base.cc:1034] running on GCE node
I20260924 07:14:06.564839 11145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:06.565071 11145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:06.566212 11145 hybrid_clock.cc:648] HybridClock initialized: now 1790234046566196 us; error 27 us; skew 500 ppm
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[10770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234046, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:06.568674 11145 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:14:06.569061 11145 webserver.cc:460] Webserver started at http://127.10.130.195:39981/ using document root <none> and password file <none>
I20260924 07:14:06.569243 11145 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:06.569280 11145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:06.569370 11145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:06.570322 11145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "c2bcd361890d445da7f1593947c1271d"
format_stamp: "Formatted at 2026-09-24 07:14:06 on dist-test-slave-7q7m"
server_key: "81409c08471a4c8d231d9f35711ff8fd"
server_key_iv: "efe0a070c10159f0ae1018d51cee573a"
server_key_version: "encryptionkey@0"
I20260924 07:14:06.570619 11145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "c2bcd361890d445da7f1593947c1271d"
format_stamp: "Formatted at 2026-09-24 07:14:06 on dist-test-slave-7q7m"
server_key: "81409c08471a4c8d231d9f35711ff8fd"
server_key_iv: "efe0a070c10159f0ae1018d51cee573a"
server_key_version: "encryptionkey@0"
I20260924 07:14:06.571918 11145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:14:06.572628 11160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:06.572789 11145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:06.572866 11145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "c2bcd361890d445da7f1593947c1271d"
format_stamp: "Formatted at 2026-09-24 07:14:06 on dist-test-slave-7q7m"
server_key: "81409c08471a4c8d231d9f35711ff8fd"
server_key_iv: "efe0a070c10159f0ae1018d51cee573a"
server_key_version: "encryptionkey@0"
I20260924 07:14:06.572921 11145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:06.580621 11145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:06.582341 11145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:06.582458 11145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:06.582655 11145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:06.582919 11145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:06.582947 11145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:06.582968 11145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:06.582989 11145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:06.588645 11145 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:41581
I20260924 07:14:06.588743 11273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:41581 every 8 connection(s)
I20260924 07:14:06.589023 11145 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:06.592250 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 11145
I20260924 07:14:06.592348 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:06.592509 10763 external_mini_cluster.cc:1428] Setting key ab6ab6226d3066a70937b51f5b35d2d7
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[10770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234046, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:06.598039 10855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:06.589743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:42291 (local address 127.10.130.254:41829)
0924 07:14:06.589809 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:06.589811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:06.590009 (+ 198us) server_negotiation.cc:407] Connection header received
0924 07:14:06.590483 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:06.590485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:06.590497 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:06.590552 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:06.591465 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.591942 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.593359 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.593557 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.593614 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:06.595612 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:06.595614 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:06.595614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:06.595622 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:06.596766 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:06.597145 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:06.597147 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:06.597147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:06.597201 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:06.597430 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:06.597431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:06.597431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:06.597537 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:06.597601 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:06.597870 (+ 269us) server_negotiation.cc:299] Negotiation successful
0924 07:14:06.597953 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 07:14:06.598215 11276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:06.589722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:41829 (local address 127.10.130.195:42291)
0924 07:14:06.589885 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:06.589924 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:06.590395 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:06.590611 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:06.590614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:06.591189 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:06.591391 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:06.591398 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.592066 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.592066 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:06.593280 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:06.593283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.593630 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.593631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:06.593762 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:06.594203 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:06.594211 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:06.595519 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:06.596842 (+ 1323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:06.596849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:06.596851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:06.597071 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:06.597262 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:06.597263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:06.597263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:06.597370 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:06.597606 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:06.597613 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:06.597746 (+ 133us) client_negotiation.cc:769] Sending connection context
0924 07:14:06.597833 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 07:14:06.597987 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260924 07:14:06.598672 11274 heartbeater.cc:347] Connected to a master server at 127.10.130.254:41829
I20260924 07:14:06.598753 11274 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:06.598914 11274 heartbeater.cc:511] Master 127.10.130.254:41829 requested a full tablet report, sending...
I20260924 07:14:06.599296 10815 ts_manager.cc:194] Registered new tserver with Master: c2bcd361890d445da7f1593947c1271d (127.10.130.195:41581)
I20260924 07:14:06.599671 10815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:42291
I20260924 07:14:06.604583 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[10770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234046, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[10770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234046, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.130.193@KRBTEST.COM
I20260924 07:14:06.626447 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:06.618894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42952 (local address 127.10.130.193:38717)
0924 07:14:06.619040 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:06.619045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:06.619067 (+ 22us) server_negotiation.cc:407] Connection header received
0924 07:14:06.619093 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:06.619095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:06.619111 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:06.619177 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:06.619670 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.620440 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.621186 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.621514 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.621572 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:06.623773 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:06.623782 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:06.623783 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:06.623790 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:06.625317 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:06.625656 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:06.625659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:06.625659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:06.625714 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:06.625896 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:06.625897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:06.625897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:06.626047 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:06.626189 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:06.626264 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 07:14:06.626339 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
W20260924 07:14:06.626806 10920 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42952
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[10770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[10770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.193@KRBTEST.COM
I20260924 07:14:06.647475 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:06.640895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42954 (local address 127.10.130.193:38717)
0924 07:14:06.640972 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:06.640974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:06.641017 (+ 43us) server_negotiation.cc:407] Connection header received
0924 07:14:06.641082 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:06.641083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:06.641095 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:06.641142 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:06.641636 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.642141 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.642977 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:06.643183 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:06.643229 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:06.645425 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:06.645427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:06.645427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:06.645434 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:06.646608 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:06.646904 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:06.646904 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:06.646905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:06.646952 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:06.647093 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:06.647094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:06.647094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:06.647197 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:06.647283 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:06.647343 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 07:14:06.647405 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 07:14:06.647960 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 10877
I20260924 07:14:06.653324 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 11011
I20260924 07:14:06.657255 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 11145
I20260924 07:14:06.660931 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 10783
2026-09-24T07:14:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1701 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[11288](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[11288](info): set up 2 sockets
Sep 24 07:14:06 dist-test-slave-7q7m krb5kdc[11288](info): commencing operation
krb5kdc: starting...
W20260924 07:14:07.599133 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:07 dist-test-slave-7q7m krb5kdc[11288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:14:07.705904 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39173
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:46789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:39173
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:07.783129 11301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:07.783321 11301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:07.783349 11301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:07.784510 11301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:07.784549 11301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:07.784567 11301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:07.784583 11301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:07.784597 11301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:07.786000 11301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:46789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:39173
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39173
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:07.786222 11301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:07.786432 11301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:07.791105 11307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:07.791198 11301 server_base.cc:1034] running on GCE node
W20260924 07:14:07.791111 11309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:07.791114 11306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:07.791507 11301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:07.791792 11301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:07.792932 11301 hybrid_clock.cc:648] HybridClock initialized: now 1790234047792889 us; error 66 us; skew 500 ppm
Sep 24 07:14:07 dist-test-slave-7q7m krb5kdc[11288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234047, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:07.795205 11301 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:14:07.795650 11301 webserver.cc:460] Webserver started at http://127.10.130.254:39783/ using document root <none> and password file <none>
I20260924 07:14:07.795857 11301 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:07.795904 11301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:07.796049 11301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:07.797055 11301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "e64a4caa09cd4dcd905e3fd4edd819a9"
format_stamp: "Formatted at 2026-09-24 07:14:07 on dist-test-slave-7q7m"
server_key: "010e99fd4736cce260ecab917a3c3b8c"
server_key_iv: "a444b8f3a4538d140efddf8c160b6d97"
server_key_version: "encryptionkey@0"
I20260924 07:14:07.797369 11301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "e64a4caa09cd4dcd905e3fd4edd819a9"
format_stamp: "Formatted at 2026-09-24 07:14:07 on dist-test-slave-7q7m"
server_key: "010e99fd4736cce260ecab917a3c3b8c"
server_key_iv: "a444b8f3a4538d140efddf8c160b6d97"
server_key_version: "encryptionkey@0"
I20260924 07:14:07.798569 11301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:14:07.799196 11316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:07.799361 11301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:07.799443 11301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "e64a4caa09cd4dcd905e3fd4edd819a9"
format_stamp: "Formatted at 2026-09-24 07:14:07 on dist-test-slave-7q7m"
server_key: "010e99fd4736cce260ecab917a3c3b8c"
server_key_iv: "a444b8f3a4538d140efddf8c160b6d97"
server_key_version: "encryptionkey@0"
I20260924 07:14:07.799506 11301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:07.806640 11301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:07.808313 11301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:07.808442 11301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:07.812074 11301 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:39173
I20260924 07:14:07.812111 11368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:39173 every 8 connection(s)
I20260924 07:14:07.812415 11301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:07.813165 11369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:07.815228 11369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9: Bootstrap starting.
I20260924 07:14:07.815711 11369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:07.815922 11369 log.cc:824] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:07.816556 11369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9: No bootstrap required, opened a new log
I20260924 07:14:07.817986 11369 raft_consensus.cc:348] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64a4caa09cd4dcd905e3fd4edd819a9" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39173 } }
I20260924 07:14:07.818109 11369 raft_consensus.cc:374] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:07.818130 11369 raft_consensus.cc:729] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64a4caa09cd4dcd905e3fd4edd819a9, State: Initialized, Role: FOLLOWER
I20260924 07:14:07.818235 11369 consensus_queue.cc:260] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64a4caa09cd4dcd905e3fd4edd819a9" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39173 } }
I20260924 07:14:07.818306 11369 raft_consensus.cc:388] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:07.818336 11369 raft_consensus.cc:482] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:07.818369 11369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:07.818807 11369 raft_consensus.cc:504] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64a4caa09cd4dcd905e3fd4edd819a9" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39173 } }
I20260924 07:14:07.818883 11369 leader_election.cc:302] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e64a4caa09cd4dcd905e3fd4edd819a9; no voters:
I20260924 07:14:07.818994 11369 leader_election.cc:288] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:07.819041 11372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:07.819195 11372 raft_consensus.cc:686] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [term 1 LEADER]: Becoming Leader. State: Replica: e64a4caa09cd4dcd905e3fd4edd819a9, State: Running, Role: LEADER
I20260924 07:14:07.819198 11369 sys_catalog.cc:564] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:07.819285 11372 consensus_queue.cc:237] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64a4caa09cd4dcd905e3fd4edd819a9" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39173 } }
I20260924 07:14:07.819572 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 11301
I20260924 07:14:07.819655 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:07.819813 10763 external_mini_cluster.cc:1428] Setting key 2b24b3d76d1ce6c84ac681bb501611a6
I20260924 07:14:07.820070 11374 sys_catalog.cc:455] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e64a4caa09cd4dcd905e3fd4edd819a9. Latest consensus state: current_term: 1 leader_uuid: "e64a4caa09cd4dcd905e3fd4edd819a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64a4caa09cd4dcd905e3fd4edd819a9" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39173 } } }
I20260924 07:14:07.820142 11374 sys_catalog.cc:458] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:07.820443 11382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:07.820459 11373 sys_catalog.cc:455] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e64a4caa09cd4dcd905e3fd4edd819a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64a4caa09cd4dcd905e3fd4edd819a9" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39173 } } }
I20260924 07:14:07.820537 11373 sys_catalog.cc:458] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:07.820899 11382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:07.822350 11382 catalog_manager.cc:1347] Generated new cluster ID: 4b8d048e11464dfcb27aab27ab90b914
I20260924 07:14:07.822399 11382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:14:07 dist-test-slave-7q7m krb5kdc[11288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:07.828527 11382 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:07.829169 11382 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:07.831048 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:07.820415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54152 (local address 127.10.130.254:39173)
0924 07:14:07.820679 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:07.820688 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:07.820726 (+ 38us) server_negotiation.cc:407] Connection header received
0924 07:14:07.820835 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:07.820841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:07.820891 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:07.820999 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:07.821890 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:07.823181 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:07.824232 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:07.824831 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:07.824920 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:07.827548 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:07.827552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:07.827553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:07.827575 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:07.829501 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:07.829899 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:07.829902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:07.829902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:07.830001 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:07.830200 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:07.830201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:07.830201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:07.830433 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:07.830564 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:07.830688 (+ 124us) server_negotiation.cc:299] Negotiation successful
0924 07:14:07.830822 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":70,"threads_started":1}
I20260924 07:14:07.836129 11382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e64a4caa09cd4dcd905e3fd4edd819a9: Generated new TSK 0
I20260924 07:14:07.836273 11382 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:14:07.858604 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39173
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:46789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:07.935691 11395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:07.935897 11395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:07.935920 11395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:07.937109 11395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:07.937153 11395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:07.937180 11395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:07.938632 11395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:46789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39173
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:07.938831 11395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:07.939044 11395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:07.943639 11401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:07.943701 11395 server_base.cc:1034] running on GCE node
W20260924 07:14:07.943790 11403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:07.943627 11400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:07.944060 11395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:07.944286 11395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:07.945423 11395 hybrid_clock.cc:648] HybridClock initialized: now 1790234047945411 us; error 24 us; skew 500 ppm
Sep 24 07:14:07 dist-test-slave-7q7m krb5kdc[11288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234047, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:07.947705 11395 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:14:07.948134 11395 webserver.cc:460] Webserver started at http://127.10.130.193:42313/ using document root <none> and password file <none>
I20260924 07:14:07.948328 11395 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:07.948374 11395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:07.948504 11395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:07.949468 11395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "ed237cd68d4c46549ef02e821e4f47a8"
format_stamp: "Formatted at 2026-09-24 07:14:07 on dist-test-slave-7q7m"
server_key: "da4709a54146cffaa610ba128ca8c59b"
server_key_iv: "4cac9eed1bd600e4d6478b0a955a5e3f"
server_key_version: "encryptionkey@0"
I20260924 07:14:07.949795 11395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "ed237cd68d4c46549ef02e821e4f47a8"
format_stamp: "Formatted at 2026-09-24 07:14:07 on dist-test-slave-7q7m"
server_key: "da4709a54146cffaa610ba128ca8c59b"
server_key_iv: "4cac9eed1bd600e4d6478b0a955a5e3f"
server_key_version: "encryptionkey@0"
I20260924 07:14:07.950942 11395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:14:07.951552 11410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:07.951720 11395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:07.951797 11395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "ed237cd68d4c46549ef02e821e4f47a8"
format_stamp: "Formatted at 2026-09-24 07:14:07 on dist-test-slave-7q7m"
server_key: "da4709a54146cffaa610ba128ca8c59b"
server_key_iv: "4cac9eed1bd600e4d6478b0a955a5e3f"
server_key_version: "encryptionkey@0"
I20260924 07:14:07.951861 11395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:07.965519 11395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:07.967087 11395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:07.967229 11395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:07.967430 11395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:07.967707 11395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:07.967739 11395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:07.967765 11395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:07.967782 11395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:07.973034 11395 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:44719
I20260924 07:14:07.973074 11523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:44719 every 8 connection(s)
I20260924 07:14:07.973351 11395 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
Sep 24 07:14:07 dist-test-slave-7q7m krb5kdc[11288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234047, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:07.981341 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:07.974021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:42335 (local address 127.10.130.254:39173)
0924 07:14:07.974079 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:07.974081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:07.974234 (+ 153us) server_negotiation.cc:407] Connection header received
0924 07:14:07.975077 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:07.975078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:07.975087 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:07.975138 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:07.975658 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:07.976162 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:07.977071 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:07.977252 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:07.977292 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:07.979057 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:07.979058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:07.979058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:07.979065 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:07.980297 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:07.980643 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:07.980643 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:07.980644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:07.980695 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:07.980866 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:07.980866 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:07.980866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:07.980964 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:07.981007 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:07.981214 (+ 207us) server_negotiation.cc:299] Negotiation successful
0924 07:14:07.981278 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 07:14:07.981496 11526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:07.973969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39173 (local address 127.10.130.193:42335)
0924 07:14:07.974122 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:07.974160 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:07.974998 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:07.975189 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:07.975192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:07.975418 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:07.975598 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:07.975603 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:07.976219 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:07.976220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:07.977011 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:07.977013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:07.977294 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:07.977295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:07.977378 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:07.977765 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:07.977772 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:07.978978 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:07.980359 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:07.980364 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:07.980365 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:07.980585 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:07.980742 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:07.980743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:07.980743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:07.980824 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:07.981008 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:07.981010 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:07.981117 (+ 107us) client_negotiation.cc:769] Sending connection context
0924 07:14:07.981191 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 07:14:07.981298 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260924 07:14:07.981940 11524 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39173
I20260924 07:14:07.982016 11524 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:07.982164 11524 heartbeater.cc:511] Master 127.10.130.254:39173 requested a full tablet report, sending...
I20260924 07:14:07.982486 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 11395
I20260924 07:14:07.982568 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:07.982682 11333 ts_manager.cc:194] Registered new tserver with Master: ed237cd68d4c46549ef02e821e4f47a8 (127.10.130.193:44719)
I20260924 07:14:07.982712 10763 external_mini_cluster.cc:1428] Setting key f06d238f6b6ce5d08c3a9038a682efb1
I20260924 07:14:07.983333 11333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:42335
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:14:08.004235 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39173
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:46789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:08.079313 11529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:08.079483 11529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:08.079512 11529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:08.080629 11529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:08.080667 11529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:08.080703 11529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:08.082077 11529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:46789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39173
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:08.082279 11529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:08.082466 11529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:08.086899 11535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:08.086964 11537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:08.086900 11534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:08.087080 11529 server_base.cc:1034] running on GCE node
I20260924 07:14:08.087301 11529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:08.087546 11529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:08.088693 11529 hybrid_clock.cc:648] HybridClock initialized: now 1790234048088679 us; error 29 us; skew 500 ppm
Sep 24 07:14:08 dist-test-slave-7q7m krb5kdc[11288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234048, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:08.090996 11529 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:14:08.091388 11529 webserver.cc:460] Webserver started at http://127.10.130.194:34049/ using document root <none> and password file <none>
I20260924 07:14:08.091570 11529 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:08.091615 11529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:08.091727 11529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:08.092674 11529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "f30a45d3584f458d99bc1825fb8c27b4"
format_stamp: "Formatted at 2026-09-24 07:14:08 on dist-test-slave-7q7m"
server_key: "45ded984d4aa7a5b4a0dc45e03b7d1e1"
server_key_iv: "1f4564e82cad17ee1bf53dd60245f657"
server_key_version: "encryptionkey@0"
I20260924 07:14:08.092973 11529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "f30a45d3584f458d99bc1825fb8c27b4"
format_stamp: "Formatted at 2026-09-24 07:14:08 on dist-test-slave-7q7m"
server_key: "45ded984d4aa7a5b4a0dc45e03b7d1e1"
server_key_iv: "1f4564e82cad17ee1bf53dd60245f657"
server_key_version: "encryptionkey@0"
I20260924 07:14:08.094069 11529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:08.094686 11544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:08.094830 11529 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:08.094913 11529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "f30a45d3584f458d99bc1825fb8c27b4"
format_stamp: "Formatted at 2026-09-24 07:14:08 on dist-test-slave-7q7m"
server_key: "45ded984d4aa7a5b4a0dc45e03b7d1e1"
server_key_iv: "1f4564e82cad17ee1bf53dd60245f657"
server_key_version: "encryptionkey@0"
I20260924 07:14:08.094969 11529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:08.121034 11529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:08.122740 11529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:08.122874 11529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:08.123101 11529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:08.123404 11529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:08.123437 11529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:08.123466 11529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:08.123482 11529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:08.128788 11529 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:42317
I20260924 07:14:08.128830 11657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:42317 every 8 connection(s)
I20260924 07:14:08.129148 11529 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
Sep 24 07:14:08 dist-test-slave-7q7m krb5kdc[11288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234048, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:08.137189 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.129822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:55719 (local address 127.10.130.254:39173)
0924 07:14:08.129888 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:08.129891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:08.130083 (+ 192us) server_negotiation.cc:407] Connection header received
0924 07:14:08.130933 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:08.130934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:08.130943 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:08.130991 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:08.131530 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.132024 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.132974 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.133152 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.133195 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.135019 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:08.135020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:08.135020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:08.135027 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:08.136179 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:08.136504 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:08.136505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:08.136505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:08.136551 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:08.136723 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:08.136724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:08.136724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:08.136814 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:08.136856 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:08.137059 (+ 203us) server_negotiation.cc:299] Negotiation successful
0924 07:14:08.137125 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 07:14:08.137339 11660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.129788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39173 (local address 127.10.130.194:55719)
0924 07:14:08.129961 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:08.129996 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:08.130847 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:08.131046 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:08.131050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:08.131283 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:08.131468 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.131472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.132082 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.132083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.132903 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.132912 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.133208 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.133208 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.133301 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.133687 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:08.133695 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:08.134941 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:08.136241 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:08.136245 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:08.136247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:08.136449 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:08.136597 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:08.136597 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:08.136597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:08.136682 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:08.136856 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:08.136859 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:08.136968 (+ 109us) client_negotiation.cc:769] Sending connection context
0924 07:14:08.137041 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 07:14:08.137147 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260924 07:14:08.137821 11658 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39173
I20260924 07:14:08.137898 11658 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:08.137882 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 11529
I20260924 07:14:08.137976 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:08.138083 11658 heartbeater.cc:511] Master 127.10.130.254:39173 requested a full tablet report, sending...
I20260924 07:14:08.138132 10763 external_mini_cluster.cc:1428] Setting key 6ff4f3aefe8050716027ee74299dfbcb
I20260924 07:14:08.138455 11333 ts_manager.cc:194] Registered new tserver with Master: f30a45d3584f458d99bc1825fb8c27b4 (127.10.130.194:42317)
I20260924 07:14:08.138873 11333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:55719
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:14:08.159044 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39173
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:46789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:08.235376 11663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:08.235571 11663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:08.235600 11663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:08.236732 11663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:08.236770 11663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:08.236804 11663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:08.238221 11663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:46789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39173
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:08.238435 11663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:08.238641 11663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:08.243162 11669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:08.243134 11671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:08.243141 11668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:08.243312 11663 server_base.cc:1034] running on GCE node
I20260924 07:14:08.243592 11663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:08.243808 11663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:08.244944 11663 hybrid_clock.cc:648] HybridClock initialized: now 1790234048244929 us; error 30 us; skew 500 ppm
Sep 24 07:14:08 dist-test-slave-7q7m krb5kdc[11288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234048, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:08.247187 11663 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:14:08.247589 11663 webserver.cc:460] Webserver started at http://127.10.130.195:38881/ using document root <none> and password file <none>
I20260924 07:14:08.247774 11663 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:08.247820 11663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:08.247939 11663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:08.248907 11663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "504870cb40e64662bc8781c09de12fd5"
format_stamp: "Formatted at 2026-09-24 07:14:08 on dist-test-slave-7q7m"
server_key: "60b68367ce8d3ca179c734b6f7efb27f"
server_key_iv: "c5e60be8dd8265c20a391fc9c1d70acf"
server_key_version: "encryptionkey@0"
I20260924 07:14:08.249219 11663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "504870cb40e64662bc8781c09de12fd5"
format_stamp: "Formatted at 2026-09-24 07:14:08 on dist-test-slave-7q7m"
server_key: "60b68367ce8d3ca179c734b6f7efb27f"
server_key_iv: "c5e60be8dd8265c20a391fc9c1d70acf"
server_key_version: "encryptionkey@0"
I20260924 07:14:08.250382 11663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:14:08.251089 11678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:08.251255 11663 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:08.251332 11663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "504870cb40e64662bc8781c09de12fd5"
format_stamp: "Formatted at 2026-09-24 07:14:08 on dist-test-slave-7q7m"
server_key: "60b68367ce8d3ca179c734b6f7efb27f"
server_key_iv: "c5e60be8dd8265c20a391fc9c1d70acf"
server_key_version: "encryptionkey@0"
I20260924 07:14:08.251392 11663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:08.278842 11663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:08.280773 11663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:08.280926 11663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:08.281153 11663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:08.281474 11663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:08.281514 11663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:08.281548 11663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:08.281569 11663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:08.287420 11663 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:38147
I20260924 07:14:08.287461 11791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:38147 every 8 connection(s)
I20260924 07:14:08.287765 11663 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:08.293121 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 11663
I20260924 07:14:08.293203 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:08.293313 10763 external_mini_cluster.cc:1428] Setting key 4a9ca94de4a7168b53ed1e9cddc59855
Sep 24 07:14:08 dist-test-slave-7q7m krb5kdc[11288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234048, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:08.300401 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.288594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:57893 (local address 127.10.130.254:39173)
0924 07:14:08.288655 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:08.288657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:08.288714 (+ 57us) server_negotiation.cc:407] Connection header received
0924 07:14:08.289593 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:08.289594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:08.289602 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:08.289646 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:08.290202 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.290675 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.291614 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.291813 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.291860 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.298121 (+ 6261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:08.298123 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:08.298123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:08.298130 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:08.299332 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:08.299688 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:08.299689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:08.299689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:08.299734 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:08.299919 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:08.299919 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:08.299920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:08.300031 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:08.300084 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:08.300281 (+ 197us) server_negotiation.cc:299] Negotiation successful
0924 07:14:08.300338 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 07:14:08.300586 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.288412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39173 (local address 127.10.130.195:57893)
0924 07:14:08.288598 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:08.288636 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:08.289515 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:08.289709 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:08.289713 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:08.289963 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:08.290144 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.290148 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.290741 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.290742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.291550 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.291552 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.291874 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.291874 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.291991 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.296642 (+ 4651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:08.296650 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:08.298038 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:08.299396 (+ 1358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:08.299401 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:08.299402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:08.299630 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:08.299785 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:08.299786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:08.299786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:08.299875 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:08.300069 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:08.300072 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:08.300182 (+ 110us) client_negotiation.cc:769] Sending connection context
0924 07:14:08.300262 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 07:14:08.300375 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260924 07:14:08.300990 11792 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39173
I20260924 07:14:08.301081 11792 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:08.301221 11792 heartbeater.cc:511] Master 127.10.130.254:39173 requested a full tablet report, sending...
I20260924 07:14:08.301587 11333 ts_manager.cc:194] Registered new tserver with Master: 504870cb40e64662bc8781c09de12fd5 (127.10.130.195:38147)
I20260924 07:14:08.301944 11333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:57893
I20260924 07:14:08.304926 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:14:08.311347 11386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.306378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54156 (local address 127.10.130.254:39173)
0924 07:14:08.306444 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:08.306446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:08.306491 (+ 45us) server_negotiation.cc:407] Connection header received
0924 07:14:08.306541 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:08.306542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:08.306550 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:08.306593 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:08.307067 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.307539 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.308250 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.308424 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.308468 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.309478 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:08.309479 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:08.309479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:08.309483 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:08.310487 (+ 1004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:08.310761 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:08.310761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:08.310762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:08.310803 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:08.310931 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:08.310932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:08.310932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:08.311051 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:08.311127 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:08.311215 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 07:14:08.311282 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 07:14:08.313791 11333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:14:08.315739 11333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:08.321910 11801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.317658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:44719 (local address 127.0.0.1:44142)
0924 07:14:08.317795 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:08.317833 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:08.317895 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:08.318328 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:08.318331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:08.318333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:08.318428 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.318430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.319660 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.319661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.320773 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.320776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.321621 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.321622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.321697 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.321710 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 07:14:08.321787 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 07:14:08.321836 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":22,"thread_start_us":67,"threads_started":1}
I20260924 07:14:08.321930 11805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.318014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44142 (local address 127.10.130.193:44719)
0924 07:14:08.318184 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:08.318188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:08.318196 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:08.318226 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:08.318227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:08.318239 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:08.318304 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:08.318512 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.319551 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.320845 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.321531 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.321609 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.321665 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:08.321811 (+ 146us) server_negotiation.cc:299] Negotiation successful
0924 07:14:08.321852 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":74,"threads_started":1}
I20260924 07:14:08.322039 11802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.317830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.10.130.194:42317)
0924 07:14:08.318172 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:08.318176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:08.318185 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:08.318221 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:08.318223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:08.318236 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:08.318307 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:08.318514 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.319379 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.320970 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.321649 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.321707 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.321750 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:08.321916 (+ 166us) server_negotiation.cc:299] Negotiation successful
0924 07:14:08.321961 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":79,"threads_started":1}
I20260924 07:14:08.321997 11803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.317702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:42317 (local address 127.0.0.1:34192)
0924 07:14:08.317988 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:08.317997 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:08.318045 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:08.318330 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:08.318331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:08.318337 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:08.318422 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.318424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.319497 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.319498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.320885 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.320888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.321781 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.321783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.321842 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.321845 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:08.321899 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 07:14:08.321935 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":2,"thread_start_us":50,"threads_started":1}
I20260924 07:14:08.322386 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.317979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33356 (local address 127.10.130.195:38147)
0924 07:14:08.318146 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:08.318151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:08.318480 (+ 329us) server_negotiation.cc:407] Connection header received
0924 07:14:08.318524 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:08.318526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:08.318540 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:08.318619 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:08.318828 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.319904 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.321183 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.321732 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.321798 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.321847 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:08.322282 (+ 435us) server_negotiation.cc:299] Negotiation successful
0924 07:14:08.322314 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":82,"threads_started":1}
I20260924 07:14:08.322233 11800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.317863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38147 (local address 127.0.0.1:33356)
0924 07:14:08.318399 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:08.318409 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:08.318461 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:08.318702 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:08.318703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:08.318705 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:08.318738 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.318739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.320047 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.320047 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.321090 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.321092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.322017 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.322018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.322092 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.322096 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:08.322146 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 07:14:08.322180 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":53,"threads_started":1}
I20260924 07:14:08.323050 11458 tablet_service.cc:1467] Processing CreateTablet for tablet 08d0abb4a3b7443dbd4da592a96de951 (DEFAULT_TABLE table=test-table [id=b28bbddb1a5e456d99e0c88dbd7a58e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:08.323050 11592 tablet_service.cc:1467] Processing CreateTablet for tablet 08d0abb4a3b7443dbd4da592a96de951 (DEFAULT_TABLE table=test-table [id=b28bbddb1a5e456d99e0c88dbd7a58e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:08.323132 11726 tablet_service.cc:1467] Processing CreateTablet for tablet 08d0abb4a3b7443dbd4da592a96de951 (DEFAULT_TABLE table=test-table [id=b28bbddb1a5e456d99e0c88dbd7a58e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:08.323400 11458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08d0abb4a3b7443dbd4da592a96de951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:08.323398 11592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08d0abb4a3b7443dbd4da592a96de951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:08.323427 11726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08d0abb4a3b7443dbd4da592a96de951. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:08.325899 11806 tablet_bootstrap.cc:492] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5: Bootstrap starting.
I20260924 07:14:08.326514 11807 tablet_bootstrap.cc:492] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4: Bootstrap starting.
I20260924 07:14:08.326808 11806 tablet_bootstrap.cc:654] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:08.327167 11807 tablet_bootstrap.cc:654] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:08.327417 11806 log.cc:824] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:08.327687 11807 log.cc:824] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:08.328009 11808 tablet_bootstrap.cc:492] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8: Bootstrap starting.
I20260924 07:14:08.328213 11806 tablet_bootstrap.cc:492] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5: No bootstrap required, opened a new log
I20260924 07:14:08.328277 11806 ts_tablet_manager.cc:1397] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:08.328361 11807 tablet_bootstrap.cc:492] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4: No bootstrap required, opened a new log
I20260924 07:14:08.328421 11807 ts_tablet_manager.cc:1397] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:08.329782 11806 raft_consensus.cc:348] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed237cd68d4c46549ef02e821e4f47a8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 44719 } } peers { permanent_uuid: "504870cb40e64662bc8781c09de12fd5" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38147 } } peers { permanent_uuid: "f30a45d3584f458d99bc1825fb8c27b4" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 42317 } }
I20260924 07:14:08.329919 11806 raft_consensus.cc:374] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:08.329849 11807 raft_consensus.cc:348] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed237cd68d4c46549ef02e821e4f47a8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 44719 } } peers { permanent_uuid: "504870cb40e64662bc8781c09de12fd5" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38147 } } peers { permanent_uuid: "f30a45d3584f458d99bc1825fb8c27b4" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 42317 } }
I20260924 07:14:08.329950 11806 raft_consensus.cc:729] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 504870cb40e64662bc8781c09de12fd5, State: Initialized, Role: FOLLOWER
I20260924 07:14:08.329965 11807 raft_consensus.cc:374] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:08.329994 11807 raft_consensus.cc:729] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30a45d3584f458d99bc1825fb8c27b4, State: Initialized, Role: FOLLOWER
I20260924 07:14:08.330063 11806 consensus_queue.cc:260] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed237cd68d4c46549ef02e821e4f47a8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 44719 } } peers { permanent_uuid: "504870cb40e64662bc8781c09de12fd5" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38147 } } peers { permanent_uuid: "f30a45d3584f458d99bc1825fb8c27b4" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 42317 } }
I20260924 07:14:08.330111 11807 consensus_queue.cc:260] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed237cd68d4c46549ef02e821e4f47a8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 44719 } } peers { permanent_uuid: "504870cb40e64662bc8781c09de12fd5" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38147 } } peers { permanent_uuid: "f30a45d3584f458d99bc1825fb8c27b4" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 42317 } }
I20260924 07:14:08.330333 11807 ts_tablet_manager.cc:1428] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:14:08.330334 11806 ts_tablet_manager.cc:1428] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 07:14:08.330389 11792 heartbeater.cc:503] Master 127.10.130.254:39173 was elected leader, sending a full tablet report...
I20260924 07:14:08.330466 11658 heartbeater.cc:503] Master 127.10.130.254:39173 was elected leader, sending a full tablet report...
I20260924 07:14:08.330969 11808 tablet_bootstrap.cc:654] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:08.331487 11808 log.cc:824] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:08.332201 11808 tablet_bootstrap.cc:492] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8: No bootstrap required, opened a new log
I20260924 07:14:08.332269 11808 ts_tablet_manager.cc:1397] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260924 07:14:08.333752 11808 raft_consensus.cc:348] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed237cd68d4c46549ef02e821e4f47a8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 44719 } } peers { permanent_uuid: "504870cb40e64662bc8781c09de12fd5" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38147 } } peers { permanent_uuid: "f30a45d3584f458d99bc1825fb8c27b4" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 42317 } }
I20260924 07:14:08.333885 11808 raft_consensus.cc:374] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:08.333916 11808 raft_consensus.cc:729] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed237cd68d4c46549ef02e821e4f47a8, State: Initialized, Role: FOLLOWER
I20260924 07:14:08.334028 11808 consensus_queue.cc:260] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed237cd68d4c46549ef02e821e4f47a8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 44719 } } peers { permanent_uuid: "504870cb40e64662bc8781c09de12fd5" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38147 } } peers { permanent_uuid: "f30a45d3584f458d99bc1825fb8c27b4" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 42317 } }
I20260924 07:14:08.334271 11808 ts_tablet_manager.cc:1428] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:14:08.334295 11524 heartbeater.cc:503] Master 127.10.130.254:39173 was elected leader, sending a full tablet report...
W20260924 07:14:08.379647 11659 tablet.cc:2369] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 07:14:08.474021 11525 tablet.cc:2369] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 07:14:08.538352 11793 tablet.cc:2369] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:14:08.586056 11812 raft_consensus.cc:482] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:08.586179 11812 raft_consensus.cc:504] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed237cd68d4c46549ef02e821e4f47a8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 44719 } } peers { permanent_uuid: "504870cb40e64662bc8781c09de12fd5" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38147 } } peers { permanent_uuid: "f30a45d3584f458d99bc1825fb8c27b4" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 42317 } }
I20260924 07:14:08.586527 11812 leader_election.cc:288] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed237cd68d4c46549ef02e821e4f47a8 (127.10.130.193:44719), 504870cb40e64662bc8781c09de12fd5 (127.10.130.195:38147)
I20260924 07:14:08.589473 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.586891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:38359 (local address 127.10.130.195:38147)
0924 07:14:08.586948 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:08.586950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:08.587047 (+ 97us) server_negotiation.cc:407] Connection header received
0924 07:14:08.587082 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:08.587083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:08.587092 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:08.587170 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:08.587310 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.587778 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.588792 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.589159 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.589226 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.589233 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:08.589374 (+ 141us) server_negotiation.cc:299] Negotiation successful
0924 07:14:08.589405 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 07:14:08.589473 11805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.586801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:58161 (local address 127.10.130.193:44719)
0924 07:14:08.586849 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:08.586851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:08.586861 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:14:08.586924 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:08.586924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:08.586933 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:08.586985 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:08.587186 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.587720 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.588789 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.589132 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.589185 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.589193 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:08.589356 (+ 163us) server_negotiation.cc:299] Negotiation successful
0924 07:14:08.589394 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 07:14:08.589483 11660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.586703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:44719 (local address 127.10.130.194:58161)
0924 07:14:08.586762 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:08.586779 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:08.586851 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:08.587014 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:08.587015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:08.587017 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:08.587098 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.587100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.587799 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.587799 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.588698 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.588701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.589213 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.589214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.589286 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.589289 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:08.589333 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 07:14:08.589370 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":3}
I20260924 07:14:08.589493 11815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.586803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38147 (local address 127.10.130.194:38359)
0924 07:14:08.586985 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:08.586994 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:08.587041 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:08.587183 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:08.587184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:08.587186 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:08.587251 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.587253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.587859 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.587860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.588702 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:08.588705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.589245 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.589246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:08.589307 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.589310 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:08.589358 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 07:14:08.589394 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260924 07:14:08.589808 11746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08d0abb4a3b7443dbd4da592a96de951" candidate_uuid: "f30a45d3584f458d99bc1825fb8c27b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "504870cb40e64662bc8781c09de12fd5" is_pre_election: true
I20260924 07:14:08.589959 11746 raft_consensus.cc:2454] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f30a45d3584f458d99bc1825fb8c27b4 in term 0.
I20260924 07:14:08.589962 11478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08d0abb4a3b7443dbd4da592a96de951" candidate_uuid: "f30a45d3584f458d99bc1825fb8c27b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed237cd68d4c46549ef02e821e4f47a8" is_pre_election: true
I20260924 07:14:08.590112 11478 raft_consensus.cc:2454] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f30a45d3584f458d99bc1825fb8c27b4 in term 0.
I20260924 07:14:08.590193 11547 leader_election.cc:302] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 504870cb40e64662bc8781c09de12fd5, f30a45d3584f458d99bc1825fb8c27b4; no voters:
I20260924 07:14:08.590313 11812 raft_consensus.cc:2781] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:08.590380 11812 raft_consensus.cc:482] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:08.590399 11812 raft_consensus.cc:3037] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:08.591037 11812 raft_consensus.cc:504] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed237cd68d4c46549ef02e821e4f47a8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 44719 } } peers { permanent_uuid: "504870cb40e64662bc8781c09de12fd5" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38147 } } peers { permanent_uuid: "f30a45d3584f458d99bc1825fb8c27b4" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 42317 } }
I20260924 07:14:08.591143 11812 leader_election.cc:288] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [CANDIDATE]: Term 1 election: Requested vote from peers ed237cd68d4c46549ef02e821e4f47a8 (127.10.130.193:44719), 504870cb40e64662bc8781c09de12fd5 (127.10.130.195:38147)
I20260924 07:14:08.591315 11478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08d0abb4a3b7443dbd4da592a96de951" candidate_uuid: "f30a45d3584f458d99bc1825fb8c27b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed237cd68d4c46549ef02e821e4f47a8"
I20260924 07:14:08.591322 11746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08d0abb4a3b7443dbd4da592a96de951" candidate_uuid: "f30a45d3584f458d99bc1825fb8c27b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "504870cb40e64662bc8781c09de12fd5"
I20260924 07:14:08.591378 11478 raft_consensus.cc:3037] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:08.591390 11746 raft_consensus.cc:3037] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:08.592087 11478 raft_consensus.cc:2454] T 08d0abb4a3b7443dbd4da592a96de951 P ed237cd68d4c46549ef02e821e4f47a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f30a45d3584f458d99bc1825fb8c27b4 in term 1.
I20260924 07:14:08.592106 11746 raft_consensus.cc:2454] T 08d0abb4a3b7443dbd4da592a96de951 P 504870cb40e64662bc8781c09de12fd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f30a45d3584f458d99bc1825fb8c27b4 in term 1.
I20260924 07:14:08.592229 11547 leader_election.cc:302] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed237cd68d4c46549ef02e821e4f47a8, f30a45d3584f458d99bc1825fb8c27b4; no voters:
I20260924 07:14:08.592303 11812 raft_consensus.cc:2781] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:08.592442 11812 raft_consensus.cc:686] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [term 1 LEADER]: Becoming Leader. State: Replica: f30a45d3584f458d99bc1825fb8c27b4, State: Running, Role: LEADER
I20260924 07:14:08.592530 11812 consensus_queue.cc:237] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed237cd68d4c46549ef02e821e4f47a8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 44719 } } peers { permanent_uuid: "504870cb40e64662bc8781c09de12fd5" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38147 } } peers { permanent_uuid: "f30a45d3584f458d99bc1825fb8c27b4" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 42317 } }
I20260924 07:14:08.593281 11333 catalog_manager.cc:5495] T 08d0abb4a3b7443dbd4da592a96de951 P f30a45d3584f458d99bc1825fb8c27b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to f30a45d3584f458d99bc1825fb8c27b4 (127.10.130.194). New cstate: current_term: 1 leader_uuid: "f30a45d3584f458d99bc1825fb8c27b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed237cd68d4c46549ef02e821e4f47a8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 44719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "504870cb40e64662bc8781c09de12fd5" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f30a45d3584f458d99bc1825fb8c27b4" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 42317 } health_report { overall_health: HEALTHY } } }
Sep 24 07:14:08 dist-test-slave-7q7m krb5kdc[11288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234047, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.193@KRBTEST.COM
I20260924 07:14:08.608922 11805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.602238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44156 (local address 127.10.130.193:44719)
0924 07:14:08.602296 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:08.602298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:08.602435 (+ 137us) server_negotiation.cc:407] Connection header received
0924 07:14:08.602494 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:08.602495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:08.602506 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:08.602553 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:08.603089 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.603547 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.604292 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.604478 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.604532 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.606703 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:08.606715 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:08.606715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:08.606723 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:08.608003 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:08.608320 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:08.608323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:08.608324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:08.608388 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:08.608528 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:08.608528 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:08.608529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:08.608652 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:08.608747 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:08.608795 (+ 48us) server_negotiation.cc:299] Negotiation successful
0924 07:14:08.608855 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Sep 24 07:14:08 dist-test-slave-7q7m krb5kdc[11288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234048, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 07:14:08 dist-test-slave-7q7m krb5kdc[11288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234048, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.130.193@KRBTEST.COM
I20260924 07:14:08.629673 11805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.623544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44158 (local address 127.10.130.193:44719)
0924 07:14:08.623605 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:08.623607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:08.623660 (+ 53us) server_negotiation.cc:407] Connection header received
0924 07:14:08.623725 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:08.623727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:08.623737 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:08.623783 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:08.624331 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.624786 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.625490 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.625677 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.625732 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.627728 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:08.627729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:08.627730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:08.627736 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:08.628824 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:08.629116 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:08.629116 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:08.629117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:08.629160 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:08.629292 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:08.629293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:08.629293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:08.629371 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:08.629432 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:08.629548 (+ 116us) server_negotiation.cc:299] Negotiation successful
0924 07:14:08.629605 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Sep 24 07:14:08 dist-test-slave-7q7m krb5kdc[11288](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 07:14:08 dist-test-slave-7q7m krb5kdc[11288](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234048, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.193@KRBTEST.COM
I20260924 07:14:08.649911 11805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:08.643337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44162 (local address 127.10.130.193:44719)
0924 07:14:08.643397 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:08.643399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:08.643455 (+ 56us) server_negotiation.cc:407] Connection header received
0924 07:14:08.643521 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:08.643522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:08.643532 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:08.643586 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:08.644112 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.644579 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.645285 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:08.645464 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:08.645517 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:08.647587 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:08.647588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:08.647589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:08.647595 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:08.648793 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:08.649192 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:08.649194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:08.649194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:08.649250 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:08.649427 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:08.649428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:08.649428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:08.649528 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:08.649613 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:08.649745 (+ 132us) server_negotiation.cc:299] Negotiation successful
0924 07:14:08.649819 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260924 07:14:08.650346 11438 tablet_service.cc:2908] Rejecting scan request for tablet 08d0abb4a3b7443dbd4da592a96de951: Uninitialized: clean time has not yet been initialized
I20260924 07:14:08.651090 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 11395
I20260924 07:14:08.655424 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 11529
I20260924 07:14:08.659188 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 11663
I20260924 07:14:08.662705 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 11301
2026-09-24T07:14:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2001 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:08 dist-test-slave-7q7m krb5kdc[11830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:08 dist-test-slave-7q7m krb5kdc[11830](info): set up 2 sockets
Sep 24 07:14:08 dist-test-slave-7q7m krb5kdc[11830](info): commencing operation
krb5kdc: starting...
W20260924 07:14:09.603252 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:09 dist-test-slave-7q7m krb5kdc[11830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:14:09.709931 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:33301
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:33301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:09.787051 11843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:09.787269 11843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:09.787289 11843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:09.788448 11843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:09.788488 11843 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 07:14:09.788501 11843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:09.788513 11843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:09.788537 11843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:09.788553 11843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:09.790072 11843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:33301
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:33301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:09.790297 11843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:09.790494 11843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:09.795027 11848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:09.795092 11843 server_base.cc:1034] running on GCE node
W20260924 07:14:09.795065 11851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:09.795032 11849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:09.795495 11843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:09.795755 11843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:09.796882 11843 hybrid_clock.cc:648] HybridClock initialized: now 1790234049796869 us; error 30 us; skew 500 ppm
Sep 24 07:14:09 dist-test-slave-7q7m krb5kdc[11830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234049, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:09.799069 11843 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:14:09.799484 11843 webserver.cc:460] Webserver started at http://127.10.130.254:46449/ using document root <none> and password file <none>
I20260924 07:14:09.799687 11843 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:09.799734 11843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:09.799851 11843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:09.800890 11843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "9a981328553c4f5ba60e7b8d7fc782c6"
format_stamp: "Formatted at 2026-09-24 07:14:09 on dist-test-slave-7q7m"
server_key: "ed6b249bdb5f2586dbc7f48c068eb9c5"
server_key_iv: "b5bfce640af8d5d34c2218ed2437267c"
server_key_version: "encryptionkey@0"
I20260924 07:14:09.801221 11843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "9a981328553c4f5ba60e7b8d7fc782c6"
format_stamp: "Formatted at 2026-09-24 07:14:09 on dist-test-slave-7q7m"
server_key: "ed6b249bdb5f2586dbc7f48c068eb9c5"
server_key_iv: "b5bfce640af8d5d34c2218ed2437267c"
server_key_version: "encryptionkey@0"
I20260924 07:14:09.802628 11843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:09.803299 11858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:09.803493 11843 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:09.803592 11843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "9a981328553c4f5ba60e7b8d7fc782c6"
format_stamp: "Formatted at 2026-09-24 07:14:09 on dist-test-slave-7q7m"
server_key: "ed6b249bdb5f2586dbc7f48c068eb9c5"
server_key_iv: "b5bfce640af8d5d34c2218ed2437267c"
server_key_version: "encryptionkey@0"
I20260924 07:14:09.803650 11843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:09.815616 11843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:09.817584 11843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:09.817687 11843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:09.822057 11843 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:33301
I20260924 07:14:09.822134 11920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:33301 every 8 connection(s)
I20260924 07:14:09.822396 11843 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:09.823055 11921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:09.823851 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 11843
I20260924 07:14:09.823923 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:09.824093 10763 external_mini_cluster.cc:1428] Setting key c7410eb1f1750facf1eddea62ca493ef
I20260924 07:14:09.825788 11921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6: Bootstrap starting.
I20260924 07:14:09.826326 11921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:09.826584 11921 log.cc:824] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:09.827200 11921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6: No bootstrap required, opened a new log
I20260924 07:14:09.828511 11921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a981328553c4f5ba60e7b8d7fc782c6" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33301 } }
I20260924 07:14:09.828640 11921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:09.828671 11921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a981328553c4f5ba60e7b8d7fc782c6, State: Initialized, Role: FOLLOWER
I20260924 07:14:09.828794 11921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a981328553c4f5ba60e7b8d7fc782c6" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33301 } }
I20260924 07:14:09.828866 11921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:09.828896 11921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:09.828943 11921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:09.829434 11921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a981328553c4f5ba60e7b8d7fc782c6" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33301 } }
I20260924 07:14:09.829527 11921 leader_election.cc:302] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a981328553c4f5ba60e7b8d7fc782c6; no voters:
I20260924 07:14:09.829677 11921 leader_election.cc:288] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:09.829721 11926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:09.829881 11926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [term 1 LEADER]: Becoming Leader. State: Replica: 9a981328553c4f5ba60e7b8d7fc782c6, State: Running, Role: LEADER
I20260924 07:14:09.829900 11921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [sys.catalog]: configured and running, proceeding with master startup.
Sep 24 07:14:09 dist-test-slave-7q7m krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234049, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:09.831171 11926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a981328553c4f5ba60e7b8d7fc782c6" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33301 } }
I20260924 07:14:09.831722 11928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a981328553c4f5ba60e7b8d7fc782c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a981328553c4f5ba60e7b8d7fc782c6" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33301 } } }
I20260924 07:14:09.831879 11928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:09.831704 11927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a981328553c4f5ba60e7b8d7fc782c6. Latest consensus state: current_term: 1 leader_uuid: "9a981328553c4f5ba60e7b8d7fc782c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a981328553c4f5ba60e7b8d7fc782c6" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33301 } } }
I20260924 07:14:09.832032 11927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:09.832212 11941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:09.832674 11941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:09.834200 11941 catalog_manager.cc:1347] Generated new cluster ID: 282dac23bbba4cbeb8aaf41f26abe078
I20260924 07:14:09.834265 11941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:09.834410 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:09.824562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44742 (local address 127.10.130.254:33301)
0924 07:14:09.824725 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:09.824732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:09.824756 (+ 24us) server_negotiation.cc:407] Connection header received
0924 07:14:09.825155 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:09.825161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:09.825206 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:09.825326 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:09.825911 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:09.826845 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:09.827626 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:09.828456 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:09.828552 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:09.830969 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:09.830974 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:09.830974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:09.830986 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:09.832879 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:09.833290 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:09.833293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:09.833293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:09.833380 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:09.833589 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:09.833589 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:09.833590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:09.833815 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:09.833948 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:09.834062 (+ 114us) server_negotiation.cc:299] Negotiation successful
0924 07:14:09.834189 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260924 07:14:09.850977 11941 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:09.851701 11941 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:09.856863 11941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6: Generated new TSK 0
I20260924 07:14:09.857056 11941 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:14:09.882840 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:33301
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:09.960140 11947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:09.960337 11947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:09.960369 11947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:09.961490 11947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:09.961529 11947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:09.961555 11947 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 07:14:09.961578 11947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:09.962945 11947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.130.254:33301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:09.963172 11947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:09.963383 11947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:09.967998 11952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:09.968072 11955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:09.968098 11953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:09.968152 11947 server_base.cc:1034] running on GCE node
I20260924 07:14:09.968457 11947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:09.968657 11947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:09.969816 11947 hybrid_clock.cc:648] HybridClock initialized: now 1790234049969800 us; error 28 us; skew 500 ppm
Sep 24 07:14:09 dist-test-slave-7q7m krb5kdc[11830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234049, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:09.972179 11947 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:14:09.972574 11947 webserver.cc:460] Webserver started at http://127.10.130.193:34863/ using document root <none> and password file <none>
I20260924 07:14:09.972743 11947 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:09.972779 11947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:09.972862 11947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:09.973807 11947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "4c0062cd8c25408b9c80fdfad9a795f0"
format_stamp: "Formatted at 2026-09-24 07:14:09 on dist-test-slave-7q7m"
server_key: "ac7a6b14412a5c83c16cea0539f84957"
server_key_iv: "136175579eaeed836a78b0c19ec2549e"
server_key_version: "encryptionkey@0"
I20260924 07:14:09.974093 11947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "4c0062cd8c25408b9c80fdfad9a795f0"
format_stamp: "Formatted at 2026-09-24 07:14:09 on dist-test-slave-7q7m"
server_key: "ac7a6b14412a5c83c16cea0539f84957"
server_key_iv: "136175579eaeed836a78b0c19ec2549e"
server_key_version: "encryptionkey@0"
I20260924 07:14:09.975195 11947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:14:09.975838 11962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:09.976046 11947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 07:14:09.976119 11947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "4c0062cd8c25408b9c80fdfad9a795f0"
format_stamp: "Formatted at 2026-09-24 07:14:09 on dist-test-slave-7q7m"
server_key: "ac7a6b14412a5c83c16cea0539f84957"
server_key_iv: "136175579eaeed836a78b0c19ec2549e"
server_key_version: "encryptionkey@0"
I20260924 07:14:09.976190 11947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:10.007838 11947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:10.009902 11947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:10.010032 11947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:10.010535 11947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:10.010581 11947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:10.010612 11947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:10.010628 11947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:10.016831 11947 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:40945
I20260924 07:14:10.016903 12077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:40945 every 8 connection(s)
Sep 24 07:14:10 dist-test-slave-7q7m krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234049, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:10.017211 11947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:14:10.017910 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 11947
I20260924 07:14:10.018015 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:10.018189 10763 external_mini_cluster.cc:1428] Setting key 8650413e6b0076a9eb46c02f13d2637d
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
I20260924 07:14:10.025611 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.011012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:46501 (local address 127.10.130.254:33301)
0924 07:14:10.011067 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.011069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.011269 (+ 200us) server_negotiation.cc:407] Connection header received
0924 07:14:10.011807 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.011809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.011823 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.011896 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:10.012766 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.013531 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.014982 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.015290 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.015370 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.017722 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:10.017724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:10.017724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:10.017730 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:10.023897 (+ 6167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.024469 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.024479 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.024479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.024536 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.024886 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.024887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.024887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.025014 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:10.025072 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.025431 (+ 359us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.025541 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 07:14:10.025728 11971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.011009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:33301 (local address 127.10.130.193:46501)
0924 07:14:10.011150 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.011186 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.011681 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.012000 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.012005 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.012350 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:10.012653 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.012660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.013625 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.013625 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.014886 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.014889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.015373 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.015373 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.015510 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.016083 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:10.016091 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:10.017631 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:10.024031 (+ 6400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.024042 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.024045 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.024381 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.024608 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.024609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.024609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.024776 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.025088 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:10.025095 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:10.025285 (+ 190us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.025397 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.025543 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":79,"threads_started":1}
I20260924 07:14:10.026232 12078 heartbeater.cc:347] Connected to a master server at 127.10.130.254:33301
I20260924 07:14:10.026324 12078 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:10.026521 12078 heartbeater.cc:511] Master 127.10.130.254:33301 requested a full tablet report, sending...
I20260924 07:14:10.027051 11874 ts_manager.cc:194] Registered new tserver with Master: 4c0062cd8c25408b9c80fdfad9a795f0 (127.10.130.193:40945)
I20260924 07:14:10.027930 11874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:46501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
I20260924 07:14:10.032938 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.027342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:45159 (local address 127.10.130.254:33301)
0924 07:14:10.027381 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.027383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.027440 (+ 57us) server_negotiation.cc:407] Connection header received
0924 07:14:10.027506 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.027506 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.027514 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.027557 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:10.027892 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.028597 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.029320 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.029518 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.029576 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.030605 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:10.030606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:10.030606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:10.030612 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:10.031838 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.032249 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.032250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.032250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.032305 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.032517 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.032518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.032518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.032611 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:10.032659 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.032821 (+ 162us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.032879 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 07:14:10.033010 12085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.027264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:33301 (local address 127.10.130.193:45159)
0924 07:14:10.027384 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.027395 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.027456 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.027587 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.027587 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.027768 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:10.027833 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.027835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.028697 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.028699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.029236 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.029238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.029587 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.029588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.029653 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.030109 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:10.030113 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:10.030537 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:10.031921 (+ 1384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.031922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.031922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.032152 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.032381 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.032382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.032382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.032448 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.032672 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:10.032673 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:10.032730 (+ 57us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.032805 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.032905 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":102,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:14:10.042722 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:33301
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:10.121167 12087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:10.121378 12087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:10.121413 12087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:10.122597 12087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:10.122642 12087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:10.122670 12087 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 07:14:10.122692 12087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:10.124155 12087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.130.254:33301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:10.124400 12087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:10.124620 12087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:10.129544 12092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:10.129544 12095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:10.129614 12093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:10.129861 12087 server_base.cc:1034] running on GCE node
I20260924 07:14:10.130132 12087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:10.130395 12087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:10.131605 12087 hybrid_clock.cc:648] HybridClock initialized: now 1790234050131586 us; error 32 us; skew 500 ppm
Sep 24 07:14:10 dist-test-slave-7q7m krb5kdc[11830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234050, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:10.133913 12087 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:14:10.134321 12087 webserver.cc:460] Webserver started at http://127.10.130.194:38075/ using document root <none> and password file <none>
I20260924 07:14:10.134552 12087 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:10.134600 12087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:10.134717 12087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:10.135723 12087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7"
format_stamp: "Formatted at 2026-09-24 07:14:10 on dist-test-slave-7q7m"
server_key: "9f8fcad4e16dd49ca058caea97bfe297"
server_key_iv: "c59af1c1a99a152fc29fe33c304b8e80"
server_key_version: "encryptionkey@0"
I20260924 07:14:10.136085 12087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7"
format_stamp: "Formatted at 2026-09-24 07:14:10 on dist-test-slave-7q7m"
server_key: "9f8fcad4e16dd49ca058caea97bfe297"
server_key_iv: "c59af1c1a99a152fc29fe33c304b8e80"
server_key_version: "encryptionkey@0"
I20260924 07:14:10.137292 12087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:10.137954 12102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:10.138113 12087 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:10.138192 12087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7"
format_stamp: "Formatted at 2026-09-24 07:14:10 on dist-test-slave-7q7m"
server_key: "9f8fcad4e16dd49ca058caea97bfe297"
server_key_iv: "c59af1c1a99a152fc29fe33c304b8e80"
server_key_version: "encryptionkey@0"
I20260924 07:14:10.138254 12087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:10.150326 12087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:10.152020 12087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:10.152158 12087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:10.152653 12087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:10.152693 12087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:10.152720 12087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:10.152735 12087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:10.158833 12087 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:43375
Sep 24 07:14:10 dist-test-slave-7q7m krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234050, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:10.158896 12217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:43375 every 8 connection(s)
I20260924 07:14:10.159179 12087 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:14:10.161732 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.153066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:50223 (local address 127.10.130.254:33301)
0924 07:14:10.153116 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.153118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.153302 (+ 184us) server_negotiation.cc:407] Connection header received
0924 07:14:10.153775 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.153777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.153790 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.153858 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:10.154638 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.155384 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.156710 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.156934 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.156995 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.159364 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:10.159366 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:10.159366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:10.159374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:10.160499 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.160864 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.160865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.160866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.160920 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.161144 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.161145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.161145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.161253 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:10.161307 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.161569 (+ 262us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.161645 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 07:14:10.161893 12111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.153060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:33301 (local address 127.10.130.194:50223)
0924 07:14:10.153195 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.153228 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.153675 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.153947 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.153951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.154269 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:10.154531 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.154538 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.155493 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.155494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.156627 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.156630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.157012 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.157014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.157159 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.157622 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:10.157631 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:10.159279 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:10.160576 (+ 1297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.160581 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.160583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.160791 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.160979 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.160979 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.160980 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.161083 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.161312 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:10.161318 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:10.161445 (+ 127us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.161532 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.161686 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260924 07:14:10.162333 12218 heartbeater.cc:347] Connected to a master server at 127.10.130.254:33301
I20260924 07:14:10.162405 12218 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:10.162563 12218 heartbeater.cc:511] Master 127.10.130.254:33301 requested a full tablet report, sending...
I20260924 07:14:10.162885 11875 ts_manager.cc:194] Registered new tserver with Master: 86c1a41d6d8c48a3b14d6362fb4a5fe7 (127.10.130.194:43375)
I20260924 07:14:10.163232 11875 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:50223
I20260924 07:14:10.167122 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 12087
I20260924 07:14:10.167217 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:10.167379 10763 external_mini_cluster.cc:1428] Setting key b5a5e0fecb47feb68a72e0c0bd95c8bd
I20260924 07:14:10.168001 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.163293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:36335 (local address 127.10.130.254:33301)
0924 07:14:10.163332 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.163334 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.163433 (+ 99us) server_negotiation.cc:407] Connection header received
0924 07:14:10.163492 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.163493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.163503 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.163559 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:10.163830 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.164347 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.165047 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.165222 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.165269 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.165941 (+ 672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:10.165942 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:10.165943 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:10.165948 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:10.166927 (+ 979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.167235 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.167236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.167236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.167293 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.167451 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.167452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.167452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.167624 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:10.167683 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.167828 (+ 145us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.167892 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:14:10.168002 12224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.163213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:33301 (local address 127.10.130.194:36335)
0924 07:14:10.163369 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.163379 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.163455 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.163567 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.163567 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.163704 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:10.163763 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.163765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.164471 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.164472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.164980 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.164981 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.165279 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.165279 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.165328 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.165640 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:10.165644 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:10.165877 (+ 233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:10.166985 (+ 1108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.166986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.166986 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.167129 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.167341 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.167342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.167342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.167395 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.167690 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:10.167691 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:10.167732 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.167793 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.167882 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":60,"threads_started":1}
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:14:10.188760 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:33301
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:46007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:10.265177 12227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:10.265357 12227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:10.265379 12227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:10.266503 12227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:10.266544 12227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:10.266564 12227 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 07:14:10.266592 12227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:10.268047 12227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:46007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.130.254:33301
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:10.268258 12227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:10.268460 12227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:10.273030 12235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:10.273096 12232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:10.273171 12227 server_base.cc:1034] running on GCE node
W20260924 07:14:10.273037 12233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:10.273463 12227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:10.273679 12227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:10.274808 12227 hybrid_clock.cc:648] HybridClock initialized: now 1790234050274796 us; error 27 us; skew 500 ppm
Sep 24 07:14:10 dist-test-slave-7q7m krb5kdc[11830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234050, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:10.277179 12227 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:14:10.277580 12227 webserver.cc:460] Webserver started at http://127.10.130.195:35161/ using document root <none> and password file <none>
I20260924 07:14:10.277751 12227 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:10.277788 12227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:10.277874 12227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:10.278789 12227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "adf898a555134660a839f74a62d13464"
format_stamp: "Formatted at 2026-09-24 07:14:10 on dist-test-slave-7q7m"
server_key: "e6f6de4145169d4b92a1ea300b505a8c"
server_key_iv: "0c99f35bee2cf8ebb5878fc24fbdfaeb"
server_key_version: "encryptionkey@0"
I20260924 07:14:10.279081 12227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "adf898a555134660a839f74a62d13464"
format_stamp: "Formatted at 2026-09-24 07:14:10 on dist-test-slave-7q7m"
server_key: "e6f6de4145169d4b92a1ea300b505a8c"
server_key_iv: "0c99f35bee2cf8ebb5878fc24fbdfaeb"
server_key_version: "encryptionkey@0"
I20260924 07:14:10.280217 12227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:10.280781 12242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:10.280922 12227 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:10.280985 12227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "adf898a555134660a839f74a62d13464"
format_stamp: "Formatted at 2026-09-24 07:14:10 on dist-test-slave-7q7m"
server_key: "e6f6de4145169d4b92a1ea300b505a8c"
server_key_iv: "0c99f35bee2cf8ebb5878fc24fbdfaeb"
server_key_version: "encryptionkey@0"
I20260924 07:14:10.281041 12227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:10.291683 12227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:10.293591 12227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:10.293689 12227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:10.294116 12227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:10.294154 12227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:10.294173 12227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:10.294184 12227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 07:14:10 dist-test-slave-7q7m krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234050, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:10.300707 12227 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:39829
I20260924 07:14:10.300792 12357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:39829 every 8 connection(s)
I20260924 07:14:10.301136 12227 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:10.302385 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.294482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:58415 (local address 127.10.130.254:33301)
0924 07:14:10.294521 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.294522 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.294722 (+ 200us) server_negotiation.cc:407] Connection header received
0924 07:14:10.295176 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.295177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.295192 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.295266 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:10.295891 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.296442 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.297430 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.297629 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.297686 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.299685 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:10.299686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:10.299686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:10.299695 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:10.301276 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.301712 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.301713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.301713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.301771 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.301952 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.301952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.301952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.302044 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:10.302086 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.302271 (+ 185us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.302325 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:14:10.302533 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 12227
I20260924 07:14:10.302590 12251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.294471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:33301 (local address 127.10.130.195:58415)
0924 07:14:10.294619 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.294651 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.295069 (+ 418us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.295322 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.295326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.295594 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:10.295803 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.295807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.296530 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.296531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.297340 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.297343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.297702 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.297703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.297821 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.298243 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:10.298251 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:10.299593 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:10.301368 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.301372 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.301374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.301637 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.301825 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.301826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.301826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.301916 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.302077 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:10.302080 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:10.302185 (+ 105us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.302257 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.302384 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":90,"threads_started":1}
I20260924 07:14:10.302627 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:10.302800 10763 external_mini_cluster.cc:1428] Setting key ccdcf46b6f3cb761b88bc01a217a70a6
I20260924 07:14:10.303244 12358 heartbeater.cc:347] Connected to a master server at 127.10.130.254:33301
I20260924 07:14:10.303360 12358 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:10.303536 12358 heartbeater.cc:511] Master 127.10.130.254:33301 requested a full tablet report, sending...
I20260924 07:14:10.303894 11874 ts_manager.cc:194] Registered new tserver with Master: adf898a555134660a839f74a62d13464 (127.10.130.195:39829)
I20260924 07:14:10.304335 11874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:58415
I20260924 07:14:10.308732 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.304219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:36569 (local address 127.10.130.254:33301)
0924 07:14:10.304260 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.304262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.304362 (+ 100us) server_negotiation.cc:407] Connection header received
0924 07:14:10.304386 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.304387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.304396 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.304436 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:10.304788 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.305354 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.306026 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.306196 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.306241 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.306968 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:10.306969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:10.306969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:10.306974 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:10.307940 (+ 966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.308247 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.308247 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.308248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.308290 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.308414 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.308414 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.308415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.308494 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:10.308537 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.308629 (+ 92us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.308677 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:14:10.308746 12364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.304135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:33301 (local address 127.10.130.195:36569)
0924 07:14:10.304289 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.304298 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.304349 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.304473 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.304474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.304649 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:10.304714 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.304715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.305433 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.305434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.305970 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.305972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.306249 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.306249 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.306301 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.306644 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:10.306650 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:10.306896 (+ 246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:10.308036 (+ 1140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.308037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.308037 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.308194 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.308338 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.308338 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.308338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.308381 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.308536 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:10.308537 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:10.308569 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.308615 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.308682 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":53,"threads_started":1}
I20260924 07:14:10.314369 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 07:14:10 dist-test-slave-7q7m krb5kdc[11830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234050, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 07:14:10 dist-test-slave-7q7m krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234050, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:10.336064 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.329277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44750 (local address 127.10.130.254:33301)
0924 07:14:10.329335 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.329337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.329345 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:10.329449 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.329450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.329460 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.329510 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:10.330067 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.330600 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.331372 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.331572 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.331634 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.333864 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:10.333865 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:10.333865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:10.333872 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:10.335156 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.335453 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.335454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.335454 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.335522 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.335659 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.335660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.335660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.335758 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:10.335828 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.335897 (+ 69us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.335949 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 07:14:10.337584 11874 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:14:10.339051 11874 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:10.345050 12372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.340995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:43375 (local address 127.0.0.1:53750)
0924 07:14:10.341108 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.341146 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.341204 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.341447 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.341450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.341453 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:10.341521 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.341522 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.342790 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.342791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.343743 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.343745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.344737 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.344739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.344819 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.344833 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.344899 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.344974 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":22,"thread_start_us":48,"threads_started":1}
I20260924 07:14:10.345069 12373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.341066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53750 (local address 127.10.130.194:43375)
0924 07:14:10.341229 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.341234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.341245 (+ 11us) server_negotiation.cc:407] Connection header received
0924 07:14:10.341289 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.341292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.341308 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.341380 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:10.341607 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.342704 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.343827 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.344572 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.344641 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.344697 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.344939 (+ 242us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.344979 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260924 07:14:10.345114 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.341413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49866 (local address 127.10.130.193:40945)
0924 07:14:10.341597 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.341601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.341638 (+ 37us) server_negotiation.cc:407] Connection header received
0924 07:14:10.341706 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.341708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.341723 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.341794 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:10.342029 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.342673 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.343909 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.344700 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.344772 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.344834 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.344955 (+ 121us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.345013 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":92,"threads_started":1}
I20260924 07:14:10.345389 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.341515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48626 (local address 127.10.130.195:39829)
0924 07:14:10.341761 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.341765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.341781 (+ 16us) server_negotiation.cc:407] Connection header received
0924 07:14:10.341815 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.341817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.341867 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.342088 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:10.342141 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.343143 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.344311 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.344850 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.345214 (+ 364us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.345273 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.345292 (+ 19us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.345323 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":80,"threads_started":1}
I20260924 07:14:10.345150 12374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.341179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:39829 (local address 127.0.0.1:48626)
0924 07:14:10.341253 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.341264 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.341314 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.341965 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.341966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.341969 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:10.342026 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.342027 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.343232 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.343233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.344237 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.344239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.344941 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.344942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.345020 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.345023 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.345052 (+ 29us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.345088 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"thread_start_us":42,"threads_started":1}
I20260924 07:14:10.345536 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.341295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:40945 (local address 127.0.0.1:49866)
0924 07:14:10.341585 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.341595 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.341649 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.341862 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.341863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.341865 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:10.341921 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.341923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.342755 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.342756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.343840 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.343843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.344789 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.344790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.344858 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.344863 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.344906 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.345466 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":516,"thread_start_us":54,"threads_started":1}
I20260924 07:14:10.346155 12292 tablet_service.cc:1467] Processing CreateTablet for tablet 4a9cc2bc26154b19b7b8ea36562463d9 (DEFAULT_TABLE table=test-table [id=a7f523b88da3433381510c35da9bd625]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:10.346213 12152 tablet_service.cc:1467] Processing CreateTablet for tablet 4a9cc2bc26154b19b7b8ea36562463d9 (DEFAULT_TABLE table=test-table [id=a7f523b88da3433381510c35da9bd625]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:10.346452 12292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a9cc2bc26154b19b7b8ea36562463d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:10.346500 12152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a9cc2bc26154b19b7b8ea36562463d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:10.348405 12012 tablet_service.cc:1467] Processing CreateTablet for tablet 4a9cc2bc26154b19b7b8ea36562463d9 (DEFAULT_TABLE table=test-table [id=a7f523b88da3433381510c35da9bd625]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:10.348747 12012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a9cc2bc26154b19b7b8ea36562463d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:10.349295 12379 tablet_bootstrap.cc:492] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Bootstrap starting.
I20260924 07:14:10.349802 12379 tablet_bootstrap.cc:654] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:10.350032 12379 log.cc:824] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:10.350632 12379 tablet_bootstrap.cc:492] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: No bootstrap required, opened a new log
I20260924 07:14:10.350677 12379 ts_tablet_manager.cc:1397] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:10.351356 12381 tablet_bootstrap.cc:492] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0: Bootstrap starting.
I20260924 07:14:10.351558 12378 tablet_bootstrap.cc:492] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464: Bootstrap starting.
I20260924 07:14:10.351953 12381 tablet_bootstrap.cc:654] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:10.352339 12378 tablet_bootstrap.cc:654] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:10.352324 12381 log.cc:824] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:10.352391 12379 raft_consensus.cc:348] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } }
I20260924 07:14:10.352566 12379 raft_consensus.cc:374] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:10.352607 12379 raft_consensus.cc:729] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c1a41d6d8c48a3b14d6362fb4a5fe7, State: Initialized, Role: FOLLOWER
I20260924 07:14:10.352664 12378 log.cc:824] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:10.352756 12379 consensus_queue.cc:260] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } }
I20260924 07:14:10.352999 12379 ts_tablet_manager.cc:1428] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 07:14:10.353113 12218 heartbeater.cc:503] Master 127.10.130.254:33301 was elected leader, sending a full tablet report...
I20260924 07:14:10.353119 12385 raft_consensus.cc:482] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:10.353219 12385 raft_consensus.cc:504] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } }
I20260924 07:14:10.353279 12381 tablet_bootstrap.cc:492] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0: No bootstrap required, opened a new log
I20260924 07:14:10.353319 12378 tablet_bootstrap.cc:492] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464: No bootstrap required, opened a new log
I20260924 07:14:10.353331 12381 ts_tablet_manager.cc:1397] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 07:14:10.353368 12378 ts_tablet_manager.cc:1397] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:14:10.354081 12385 leader_election.cc:288] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adf898a555134660a839f74a62d13464 (127.10.130.195:39829), 4c0062cd8c25408b9c80fdfad9a795f0 (127.10.130.193:40945)
I20260924 07:14:10.355077 12378 raft_consensus.cc:348] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } }
I20260924 07:14:10.355227 12378 raft_consensus.cc:374] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:10.355257 12378 raft_consensus.cc:729] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf898a555134660a839f74a62d13464, State: Initialized, Role: FOLLOWER
I20260924 07:14:10.355156 12381 raft_consensus.cc:348] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } }
I20260924 07:14:10.355302 12381 raft_consensus.cc:374] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:10.355340 12381 raft_consensus.cc:729] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c0062cd8c25408b9c80fdfad9a795f0, State: Initialized, Role: FOLLOWER
I20260924 07:14:10.355353 12378 consensus_queue.cc:260] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } }
I20260924 07:14:10.355458 12381 consensus_queue.cc:260] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } }
I20260924 07:14:10.355567 12378 ts_tablet_manager.cc:1428] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 07:14:10.355677 12358 heartbeater.cc:503] Master 127.10.130.254:33301 was elected leader, sending a full tablet report...
I20260924 07:14:10.355685 12381 ts_tablet_manager.cc:1428] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 07:14:10.355785 12078 heartbeater.cc:503] Master 127.10.130.254:33301 was elected leader, sending a full tablet report...
I20260924 07:14:10.357123 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.353734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:56887 (local address 127.10.130.195:39829)
0924 07:14:10.353766 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.353768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.354007 (+ 239us) server_negotiation.cc:407] Connection header received
0924 07:14:10.354029 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.354030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.354039 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.354079 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:10.354351 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.355025 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.356303 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.356798 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.356855 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.356862 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.357041 (+ 179us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.357073 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260924 07:14:10.357124 12111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.353629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:39829 (local address 127.10.130.194:56887)
0924 07:14:10.353897 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.353909 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.353956 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.354228 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.354229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.354230 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:10.354282 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.354284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.355122 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.355123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.356212 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.356215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.356892 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.356893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.356971 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.356974 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.357018 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.357058 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249}
I20260924 07:14:10.357468 12312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a9cc2bc26154b19b7b8ea36562463d9" candidate_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf898a555134660a839f74a62d13464" is_pre_election: true
I20260924 07:14:10.357591 12312 raft_consensus.cc:2454] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c1a41d6d8c48a3b14d6362fb4a5fe7 in term 0.
I20260924 07:14:10.357774 12105 leader_election.cc:302] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86c1a41d6d8c48a3b14d6362fb4a5fe7, adf898a555134660a839f74a62d13464; no voters:
I20260924 07:14:10.357924 12383 raft_consensus.cc:2781] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:10.357985 12383 raft_consensus.cc:482] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:10.358008 12383 raft_consensus.cc:3037] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:10.358351 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.354031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:55085 (local address 127.10.130.193:40945)
0924 07:14:10.354063 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.354065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.355201 (+ 1136us) server_negotiation.cc:407] Connection header received
0924 07:14:10.355285 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.355285 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.355294 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.355319 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:10.355580 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.356222 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.357516 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.358003 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.358066 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.358073 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.358254 (+ 181us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.358297 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:14:10.358450 12386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.353883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:40945 (local address 127.10.130.194:55085)
0924 07:14:10.355146 (+ 1263us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.355156 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.355224 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.355413 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.355414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.355416 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:10.355499 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.355501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.356415 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.356416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.357424 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.357427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.358118 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.358120 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.358183 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.358186 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.358223 (+ 37us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.358249 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1247,"thread_start_us":66,"threads_started":1}
I20260924 07:14:10.358716 12383 raft_consensus.cc:504] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } }
I20260924 07:14:10.358798 12032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a9cc2bc26154b19b7b8ea36562463d9" candidate_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" is_pre_election: true
I20260924 07:14:10.358839 12383 leader_election.cc:288] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [CANDIDATE]: Term 1 election: Requested vote from peers adf898a555134660a839f74a62d13464 (127.10.130.195:39829), 4c0062cd8c25408b9c80fdfad9a795f0 (127.10.130.193:40945)
I20260924 07:14:10.358896 12032 raft_consensus.cc:2454] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86c1a41d6d8c48a3b14d6362fb4a5fe7 in term 0.
I20260924 07:14:10.358951 12312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a9cc2bc26154b19b7b8ea36562463d9" candidate_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf898a555134660a839f74a62d13464"
I20260924 07:14:10.358994 12031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a9cc2bc26154b19b7b8ea36562463d9" candidate_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c0062cd8c25408b9c80fdfad9a795f0"
I20260924 07:14:10.359022 12312 raft_consensus.cc:3037] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:10.359058 12031 raft_consensus.cc:3037] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:10.359625 12031 raft_consensus.cc:2454] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c1a41d6d8c48a3b14d6362fb4a5fe7 in term 1.
I20260924 07:14:10.359629 12312 raft_consensus.cc:2454] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86c1a41d6d8c48a3b14d6362fb4a5fe7 in term 1.
I20260924 07:14:10.359758 12105 leader_election.cc:302] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c0062cd8c25408b9c80fdfad9a795f0, 86c1a41d6d8c48a3b14d6362fb4a5fe7; no voters:
I20260924 07:14:10.359864 12383 raft_consensus.cc:2781] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:10.359917 12383 raft_consensus.cc:686] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 1 LEADER]: Becoming Leader. State: Replica: 86c1a41d6d8c48a3b14d6362fb4a5fe7, State: Running, Role: LEADER
I20260924 07:14:10.360003 12383 consensus_queue.cc:237] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } }
I20260924 07:14:10.360672 11875 catalog_manager.cc:5495] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86c1a41d6d8c48a3b14d6362fb4a5fe7 (127.10.130.194). New cstate: current_term: 1 leader_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } health_report { overall_health: UNKNOWN } } }
I20260924 07:14:10.365761 12373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.363721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53760 (local address 127.10.130.194:43375)
0924 07:14:10.363757 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.363759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.363790 (+ 31us) server_negotiation.cc:407] Connection header received
0924 07:14:10.363853 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.363853 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.363866 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.363914 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:10.364045 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.364450 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.365157 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.365340 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.365389 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.365543 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:10.365619 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:10.365669 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.365705 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.365719 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:14:10.367146 12031 raft_consensus.cc:1261] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 1 FOLLOWER]: Refusing update from remote peer 86c1a41d6d8c48a3b14d6362fb4a5fe7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:10.367146 12312 raft_consensus.cc:1261] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464 [term 1 FOLLOWER]: Refusing update from remote peer 86c1a41d6d8c48a3b14d6362fb4a5fe7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:10.367460 12385 consensus_queue.cc:1035] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:10.367556 12383 consensus_queue.cc:1035] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:10.369971 12393 mvcc.cc:205] Tried to move back new op lower bound from 7332798670301753344 to 7332798670275084288. Current Snapshot: MvccSnapshot[applied={T|T < 7332798670301753344}]
I20260924 07:14:10.372716 11875 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:10.372793 11875 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:10.373456 11875 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6: Sending DeleteTablet for 3 replicas of tablet 4a9cc2bc26154b19b7b8ea36562463d9
I20260924 07:14:10.373751 12152 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a9cc2bc26154b19b7b8ea36562463d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:10 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:53750
I20260924 07:14:10.373838 12292 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a9cc2bc26154b19b7b8ea36562463d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:10 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:48626
I20260924 07:14:10.374006 12012 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a9cc2bc26154b19b7b8ea36562463d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:10 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:49866
I20260924 07:14:10.373976 11875 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:14:10.374116 11875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:10.374284 12401 tablet_replica.cc:330] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: stopping tablet replica
I20260924 07:14:10.374294 12403 tablet_replica.cc:330] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0: stopping tablet replica
I20260924 07:14:10.374372 12401 raft_consensus.cc:2229] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:10.374455 12401 raft_consensus.cc:2258] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:10.374612 12402 tablet_replica.cc:330] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464: stopping tablet replica
I20260924 07:14:10.374862 12401 ts_tablet_manager.cc:1905] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:10.375485 12152 tablet_service.cc:1467] Processing CreateTablet for tablet 18aa6c8263f3471bbc0889926c63b938 (DEFAULT_TABLE table=test-table [id=fd556f04154f4ec5b8bc9ef9332337a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:10.375485 12012 tablet_service.cc:1467] Processing CreateTablet for tablet 18aa6c8263f3471bbc0889926c63b938 (DEFAULT_TABLE table=test-table [id=fd556f04154f4ec5b8bc9ef9332337a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:10.375599 12152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18aa6c8263f3471bbc0889926c63b938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:10.375599 12012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18aa6c8263f3471bbc0889926c63b938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:10.376806 12379 tablet_bootstrap.cc:492] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Bootstrap starting.
I20260924 07:14:10.377223 12379 tablet_bootstrap.cc:654] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:10.377431 12292 tablet_service.cc:1467] Processing CreateTablet for tablet 18aa6c8263f3471bbc0889926c63b938 (DEFAULT_TABLE table=test-table [id=fd556f04154f4ec5b8bc9ef9332337a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:10.377547 12292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18aa6c8263f3471bbc0889926c63b938. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:10.377849 12402 raft_consensus.cc:2229] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:10.377966 12402 raft_consensus.cc:2258] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:10.378381 12402 ts_tablet_manager.cc:1905] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:10.378852 12378 tablet_bootstrap.cc:492] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464: Bootstrap starting.
I20260924 07:14:10.378865 12381 tablet_bootstrap.cc:492] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0: Bootstrap starting.
I20260924 07:14:10.379227 12381 tablet_bootstrap.cc:654] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:10.379269 12378 tablet_bootstrap.cc:654] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:10.379599 12402 ts_tablet_manager.cc:1918] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:10.379660 12402 log.cc:1196] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/4a9cc2bc26154b19b7b8ea36562463d9
I20260924 07:14:10.379820 12401 ts_tablet_manager.cc:1918] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:10.379876 12402 ts_tablet_manager.cc:1939] T 4a9cc2bc26154b19b7b8ea36562463d9 P adf898a555134660a839f74a62d13464: Deleting consensus metadata
I20260924 07:14:10.379884 12401 log.cc:1196] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/4a9cc2bc26154b19b7b8ea36562463d9
I20260924 07:14:10.380144 12401 ts_tablet_manager.cc:1939] T 4a9cc2bc26154b19b7b8ea36562463d9 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Deleting consensus metadata
I20260924 07:14:10.380268 11861 catalog_manager.cc:4841] TS adf898a555134660a839f74a62d13464 (127.10.130.195:39829): tablet 4a9cc2bc26154b19b7b8ea36562463d9 (table test-table [id=a7f523b88da3433381510c35da9bd625]) successfully deleted
I20260924 07:14:10.380456 12403 raft_consensus.cc:2229] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:10.380477 11861 catalog_manager.cc:4841] TS 86c1a41d6d8c48a3b14d6362fb4a5fe7 (127.10.130.194:43375): tablet 4a9cc2bc26154b19b7b8ea36562463d9 (table test-table [id=a7f523b88da3433381510c35da9bd625]) successfully deleted
I20260924 07:14:10.380587 12403 raft_consensus.cc:2258] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:10.380949 12381 tablet_bootstrap.cc:492] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0: No bootstrap required, opened a new log
I20260924 07:14:10.380996 12381 ts_tablet_manager.cc:1397] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:14:10.381031 12403 ts_tablet_manager.cc:1905] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:10.381130 12381 raft_consensus.cc:348] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.381199 12381 raft_consensus.cc:374] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:10.381218 12381 raft_consensus.cc:729] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c0062cd8c25408b9c80fdfad9a795f0, State: Initialized, Role: FOLLOWER
I20260924 07:14:10.381258 12381 consensus_queue.cc:260] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.381338 12381 ts_tablet_manager.cc:1428] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:10.381763 12378 tablet_bootstrap.cc:492] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464: No bootstrap required, opened a new log
I20260924 07:14:10.381815 12378 ts_tablet_manager.cc:1397] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260924 07:14:10.381943 12378 raft_consensus.cc:348] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.382009 12378 raft_consensus.cc:374] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:10.382028 12378 raft_consensus.cc:729] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf898a555134660a839f74a62d13464, State: Initialized, Role: FOLLOWER
I20260924 07:14:10.382064 12378 consensus_queue.cc:260] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.382139 12378 ts_tablet_manager.cc:1428] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:10.382321 12403 ts_tablet_manager.cc:1918] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:10.382385 12403 log.cc:1196] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/wal/wals/4a9cc2bc26154b19b7b8ea36562463d9
I20260924 07:14:10.382602 12403 ts_tablet_manager.cc:1939] T 4a9cc2bc26154b19b7b8ea36562463d9 P 4c0062cd8c25408b9c80fdfad9a795f0: Deleting consensus metadata
I20260924 07:14:10.382736 12379 tablet_bootstrap.cc:492] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: No bootstrap required, opened a new log
I20260924 07:14:10.382777 12379 ts_tablet_manager.cc:1397] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260924 07:14:10.382900 12379 raft_consensus.cc:348] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.382966 12379 raft_consensus.cc:374] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:10.382985 12379 raft_consensus.cc:729] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c1a41d6d8c48a3b14d6362fb4a5fe7, State: Initialized, Role: FOLLOWER
I20260924 07:14:10.383026 12379 consensus_queue.cc:260] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.383068 11861 catalog_manager.cc:4841] TS 4c0062cd8c25408b9c80fdfad9a795f0 (127.10.130.193:40945): tablet 4a9cc2bc26154b19b7b8ea36562463d9 (table test-table [id=a7f523b88da3433381510c35da9bd625]) successfully deleted
I20260924 07:14:10.383098 12379 ts_tablet_manager.cc:1428] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260924 07:14:10.409639 12219 tablet.cc:2369] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:14:10.435707 12388 raft_consensus.cc:482] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:10.435791 12388 raft_consensus.cc:504] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.436102 12388 leader_election.cc:288] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86c1a41d6d8c48a3b14d6362fb4a5fe7 (127.10.130.194:43375), adf898a555134660a839f74a62d13464 (127.10.130.195:39829)
I20260924 07:14:10.439287 12377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.436507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:58935 (local address 127.10.130.195:39829)
0924 07:14:10.436544 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.436546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.436608 (+ 62us) server_negotiation.cc:407] Connection header received
0924 07:14:10.436644 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.436645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.436653 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.436697 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:10.436864 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.437551 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.438484 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.438934 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.439004 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.439012 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.439183 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.439221 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:14:10.439522 12373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.436329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:50771 (local address 127.10.130.194:43375)
0924 07:14:10.436375 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.436376 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.436387 (+ 11us) server_negotiation.cc:407] Connection header received
0924 07:14:10.436416 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.436417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.436427 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.436491 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:10.436692 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.437407 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.438698 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.439159 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.439217 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.439224 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.439444 (+ 220us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.439479 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 07:14:10.439646 11971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.436261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:43375 (local address 127.10.130.193:50771)
0924 07:14:10.436316 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.436328 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.436366 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.436548 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.436550 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.436552 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:10.436625 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.436627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.437502 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.437504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.438603 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.438606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.439252 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.439253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.439330 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.439334 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.439377 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.439414 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 07:14:10.439316 12407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.436363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:39829 (local address 127.10.130.193:58935)
0924 07:14:10.436508 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.436516 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.436602 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.436721 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.436722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.436724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:10.436797 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.436798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.437628 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.437629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.438391 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.438393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.439027 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.439028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.439101 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.439107 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.439172 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.439213 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260924 07:14:10.439924 12172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18aa6c8263f3471bbc0889926c63b938" candidate_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" is_pre_election: true
I20260924 07:14:10.440073 12172 raft_consensus.cc:2454] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c0062cd8c25408b9c80fdfad9a795f0 in term 0.
I20260924 07:14:10.440026 12312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18aa6c8263f3471bbc0889926c63b938" candidate_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf898a555134660a839f74a62d13464" is_pre_election: true
I20260924 07:14:10.440121 12312 raft_consensus.cc:2454] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c0062cd8c25408b9c80fdfad9a795f0 in term 0.
I20260924 07:14:10.440253 11965 leader_election.cc:302] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c0062cd8c25408b9c80fdfad9a795f0, 86c1a41d6d8c48a3b14d6362fb4a5fe7; no voters:
I20260924 07:14:10.440407 12388 raft_consensus.cc:2781] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:10.440438 12388 raft_consensus.cc:482] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:10.440455 12388 raft_consensus.cc:3037] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:10.441129 12388 raft_consensus.cc:504] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.441231 12388 leader_election.cc:288] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [CANDIDATE]: Term 1 election: Requested vote from peers 86c1a41d6d8c48a3b14d6362fb4a5fe7 (127.10.130.194:43375), adf898a555134660a839f74a62d13464 (127.10.130.195:39829)
I20260924 07:14:10.441457 12312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18aa6c8263f3471bbc0889926c63b938" candidate_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf898a555134660a839f74a62d13464"
I20260924 07:14:10.441546 12312 raft_consensus.cc:3037] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:10.442343 12312 raft_consensus.cc:2454] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c0062cd8c25408b9c80fdfad9a795f0 in term 1.
I20260924 07:14:10.442457 12172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18aa6c8263f3471bbc0889926c63b938" candidate_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7"
I20260924 07:14:10.442526 12172 raft_consensus.cc:3037] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:10.442514 11965 leader_election.cc:302] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c0062cd8c25408b9c80fdfad9a795f0, adf898a555134660a839f74a62d13464; no voters:
I20260924 07:14:10.442584 12388 raft_consensus.cc:2781] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:10.442682 12388 raft_consensus.cc:686] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 1 LEADER]: Becoming Leader. State: Replica: 4c0062cd8c25408b9c80fdfad9a795f0, State: Running, Role: LEADER
I20260924 07:14:10.442742 12388 consensus_queue.cc:237] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.443292 12172 raft_consensus.cc:2454] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c0062cd8c25408b9c80fdfad9a795f0 in term 1.
I20260924 07:14:10.443764 11875 catalog_manager.cc:5495] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c0062cd8c25408b9c80fdfad9a795f0 (127.10.130.193). New cstate: current_term: 1 leader_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } health_report { overall_health: UNKNOWN } } }
Sep 24 07:14:10 dist-test-slave-7q7m krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234049, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:10.480072 12415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.472647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:33301 (local address 127.0.0.1:44752)
0924 07:14:10.472806 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.472817 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.472875 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.473018 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.473019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.473292 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:10.473341 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.473343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.474231 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.474231 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.474745 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.474746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.475185 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.475186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.475222 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.475790 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:10.475795 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:10.477369 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:10.479040 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.479047 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.479049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.479253 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.479448 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:10.479448 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:10.479449 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:10.479502 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:10.479747 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:10.479754 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:10.479796 (+ 42us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.479850 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.479936 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260924 07:14:10.480155 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.472723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.10.130.254:33301)
0924 07:14:10.472780 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.472783 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.472879 (+ 96us) server_negotiation.cc:407] Connection header received
0924 07:14:10.472912 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.472913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.472923 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.473006 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:10.473405 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.474163 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.474821 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.475121 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.475188 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.477462 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:10.477463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:10.477464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:10.477471 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:10.478946 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.479316 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.479317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.479318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.479384 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.479556 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.479557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.479558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.479689 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:10.479754 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.479890 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.480022 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":34}
I20260924 07:14:10.481904 11875 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:44752:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 07:14:10.482122 11875 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:10.483412 12012 tablet_service.cc:1467] Processing CreateTablet for tablet c54ccc2a3fdc4c19b96b5ee7e580218f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=01ee7084f32a4bfc8bff833289780d3b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 07:14:10.483467 12152 tablet_service.cc:1467] Processing CreateTablet for tablet c54ccc2a3fdc4c19b96b5ee7e580218f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=01ee7084f32a4bfc8bff833289780d3b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 07:14:10.483500 12292 tablet_service.cc:1467] Processing CreateTablet for tablet c54ccc2a3fdc4c19b96b5ee7e580218f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=01ee7084f32a4bfc8bff833289780d3b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 07:14:10.483592 12152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c54ccc2a3fdc4c19b96b5ee7e580218f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:10.483592 12012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c54ccc2a3fdc4c19b96b5ee7e580218f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:10.483629 12292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c54ccc2a3fdc4c19b96b5ee7e580218f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:10.485040 12378 tablet_bootstrap.cc:492] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464: Bootstrap starting.
I20260924 07:14:10.485040 12381 tablet_bootstrap.cc:492] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0: Bootstrap starting.
I20260924 07:14:10.485414 12381 tablet_bootstrap.cc:654] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:10.485430 12378 tablet_bootstrap.cc:654] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:10.485707 12379 tablet_bootstrap.cc:492] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Bootstrap starting.
I20260924 07:14:10.486004 12379 tablet_bootstrap.cc:654] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:10.486522 12379 tablet_bootstrap.cc:492] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7: No bootstrap required, opened a new log
I20260924 07:14:10.486527 12378 tablet_bootstrap.cc:492] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464: No bootstrap required, opened a new log
I20260924 07:14:10.486572 12378 ts_tablet_manager.cc:1397] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:14:10.486572 12379 ts_tablet_manager.cc:1397] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 07:14:10.486671 12381 tablet_bootstrap.cc:492] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0: No bootstrap required, opened a new log
I20260924 07:14:10.486706 12381 ts_tablet_manager.cc:1397] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:14:10.486668 12379 raft_consensus.cc:348] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.486732 12379 raft_consensus.cc:374] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:10.486752 12379 raft_consensus.cc:729] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c1a41d6d8c48a3b14d6362fb4a5fe7, State: Initialized, Role: FOLLOWER
I20260924 07:14:10.486714 12378 raft_consensus.cc:348] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.486778 12378 raft_consensus.cc:374] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:10.486795 12378 raft_consensus.cc:729] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf898a555134660a839f74a62d13464, State: Initialized, Role: FOLLOWER
I20260924 07:14:10.486793 12379 consensus_queue.cc:260] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.486838 12378 consensus_queue.cc:260] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.486824 12381 raft_consensus.cc:348] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.486877 12381 raft_consensus.cc:374] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:10.486879 12379 ts_tablet_manager.cc:1428] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:10.486896 12381 raft_consensus.cc:729] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c0062cd8c25408b9c80fdfad9a795f0, State: Initialized, Role: FOLLOWER
I20260924 07:14:10.486915 12378 ts_tablet_manager.cc:1428] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:10.486930 12381 consensus_queue.cc:260] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.487018 12381 ts_tablet_manager.cc:1428] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:10.486989 12385 tablet_replica.cc:439] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } }
I20260924 07:14:10.487051 12385 tablet_replica.cc:442] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:10.487030 12387 tablet_replica.cc:439] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } }
I20260924 07:14:10.487098 12387 tablet_replica.cc:442] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:10.487267 12388 tablet_replica.cc:439] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } }
I20260924 07:14:10.487340 12388 tablet_replica.cc:442] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0: This TxnStatusTablet replica's current role is: FOLLOWER
W20260924 07:14:10.517968 12079 tablet.cc:2369] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 07:14:10.551685 12359 tablet.cc:2369] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:14:10.584617 12388 raft_consensus.cc:482] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:10.584707 12388 raft_consensus.cc:504] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.584826 12388 leader_election.cc:288] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86c1a41d6d8c48a3b14d6362fb4a5fe7 (127.10.130.194:43375), adf898a555134660a839f74a62d13464 (127.10.130.195:39829)
I20260924 07:14:10.585024 12172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c54ccc2a3fdc4c19b96b5ee7e580218f" candidate_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" is_pre_election: true
I20260924 07:14:10.585122 12172 raft_consensus.cc:2454] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c0062cd8c25408b9c80fdfad9a795f0 in term 0.
I20260924 07:14:10.585093 12312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c54ccc2a3fdc4c19b96b5ee7e580218f" candidate_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf898a555134660a839f74a62d13464" is_pre_election: true
I20260924 07:14:10.585201 12312 raft_consensus.cc:2454] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c0062cd8c25408b9c80fdfad9a795f0 in term 0.
I20260924 07:14:10.585274 11965 leader_election.cc:302] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c0062cd8c25408b9c80fdfad9a795f0, 86c1a41d6d8c48a3b14d6362fb4a5fe7; no voters:
I20260924 07:14:10.585367 12388 raft_consensus.cc:2781] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:10.585412 12388 raft_consensus.cc:482] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:10.585429 12388 raft_consensus.cc:3037] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:10.586054 12388 raft_consensus.cc:504] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.586158 12388 leader_election.cc:288] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [CANDIDATE]: Term 1 election: Requested vote from peers 86c1a41d6d8c48a3b14d6362fb4a5fe7 (127.10.130.194:43375), adf898a555134660a839f74a62d13464 (127.10.130.195:39829)
I20260924 07:14:10.586323 12172 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c54ccc2a3fdc4c19b96b5ee7e580218f" candidate_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7"
I20260924 07:14:10.586349 12312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c54ccc2a3fdc4c19b96b5ee7e580218f" candidate_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf898a555134660a839f74a62d13464"
I20260924 07:14:10.586390 12172 raft_consensus.cc:3037] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:10.586405 12312 raft_consensus.cc:3037] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:10.586913 12312 raft_consensus.cc:2454] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c0062cd8c25408b9c80fdfad9a795f0 in term 1.
I20260924 07:14:10.587025 12172 raft_consensus.cc:2454] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c0062cd8c25408b9c80fdfad9a795f0 in term 1.
I20260924 07:14:10.587076 11965 leader_election.cc:302] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c0062cd8c25408b9c80fdfad9a795f0, adf898a555134660a839f74a62d13464; no voters:
I20260924 07:14:10.587185 12388 raft_consensus.cc:2781] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:10.587239 12388 raft_consensus.cc:686] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [term 1 LEADER]: Becoming Leader. State: Replica: 4c0062cd8c25408b9c80fdfad9a795f0, State: Running, Role: LEADER
I20260924 07:14:10.587287 12388 consensus_queue.cc:237] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } }
I20260924 07:14:10.587563 12408 tablet_replica.cc:439] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0: TxnStatusTablet state changed. Reason: New leader 4c0062cd8c25408b9c80fdfad9a795f0. Latest consensus state: current_term: 1 leader_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } }
I20260924 07:14:10.587646 12408 tablet_replica.cc:442] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0: This TxnStatusTablet replica's current role is: LEADER
I20260924 07:14:10.587826 12420 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 07:14:10.587854 11874 catalog_manager.cc:5495] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c0062cd8c25408b9c80fdfad9a795f0 (127.10.130.193). New cstate: current_term: 1 leader_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } health_report { overall_health: UNKNOWN } } }
I20260924 07:14:10.587929 12420 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 07:14:10.603041 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.600725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49878 (local address 127.10.130.193:40945)
0924 07:14:10.600772 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.600775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.600784 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:10.600843 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.600844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.600861 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.600917 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:10.601045 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.601528 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.602265 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.602458 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.602525 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.602706 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:10.602829 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:10.602890 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.602940 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.602963 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:14:10.603052 12415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.600648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:40945 (local address 127.0.0.1:49878)
0924 07:14:10.600715 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.600727 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.600773 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.600932 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.600933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.600934 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 07:14:10.600983 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.600985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.601612 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.601613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.602186 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.602188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.602523 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.602524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.602605 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.602621 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 07:14:10.602886 (+ 265us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 07:14:10.602896 (+ 10us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.602938 (+ 42us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.602985 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 07:14:10.604146 12312 raft_consensus.cc:1261] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464 [term 1 FOLLOWER]: Refusing update from remote peer 4c0062cd8c25408b9c80fdfad9a795f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:10.604276 12172 raft_consensus.cc:1261] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 1 FOLLOWER]: Refusing update from remote peer 4c0062cd8c25408b9c80fdfad9a795f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:10.604388 12388 consensus_queue.cc:1035] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:10.604511 12408 consensus_queue.cc:1035] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:10.604921 12387 tablet_replica.cc:439] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464: TxnStatusTablet state changed. Reason: New leader 4c0062cd8c25408b9c80fdfad9a795f0. Latest consensus state: current_term: 1 leader_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } }
I20260924 07:14:10.605016 12387 tablet_replica.cc:442] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:10.605084 12385 tablet_replica.cc:439] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7: TxnStatusTablet state changed. Reason: New leader 4c0062cd8c25408b9c80fdfad9a795f0. Latest consensus state: current_term: 1 leader_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } }
I20260924 07:14:10.605186 12385 tablet_replica.cc:442] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:10.605759 12408 tablet_replica.cc:439] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } }
I20260924 07:14:10.605835 12408 tablet_replica.cc:442] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0: This TxnStatusTablet replica's current role is: LEADER
I20260924 07:14:10.605895 12392 mvcc.cc:205] Tried to move back new op lower bound from 7332798671272734720 to 7332798671205867520. Current Snapshot: MvccSnapshot[applied={T|T < 7332798671272734720}]
I20260924 07:14:10.606233 12388 tablet_replica.cc:439] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } }
I20260924 07:14:10.606302 12388 tablet_replica.cc:442] T c54ccc2a3fdc4c19b96b5ee7e580218f P 4c0062cd8c25408b9c80fdfad9a795f0: This TxnStatusTablet replica's current role is: LEADER
I20260924 07:14:10.606400 12387 tablet_replica.cc:439] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } }
I20260924 07:14:10.606474 12387 tablet_replica.cc:442] T c54ccc2a3fdc4c19b96b5ee7e580218f P adf898a555134660a839f74a62d13464: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:10.606724 12385 tablet_replica.cc:439] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c0062cd8c25408b9c80fdfad9a795f0" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40945 } } peers { permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 } } peers { permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 } } }
I20260924 07:14:10.606797 12385 tablet_replica.cc:442] T c54ccc2a3fdc4c19b96b5ee7e580218f P 86c1a41d6d8c48a3b14d6362fb4a5fe7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:10.610267 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.607944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49888 (local address 127.10.130.193:40945)
0924 07:14:10.608002 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.608011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.608020 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:10.608067 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.608068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.608077 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.608135 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:10.608223 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.608931 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.609596 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.609895 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.609956 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.610060 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:10.610091 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:10.610134 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.610181 (+ 47us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.610198 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:14:10.615118 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.612641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:40795 (local address 127.10.130.193:40945)
0924 07:14:10.612676 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.612677 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.612819 (+ 142us) server_negotiation.cc:407] Connection header received
0924 07:14:10.612840 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.612840 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.612849 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.612912 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:10.613020 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.613700 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.614423 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.614706 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.614773 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.614897 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:10.614926 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:10.614971 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.615023 (+ 52us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.615039 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:14:10.615393 12427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.612574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:40945 (local address 127.10.130.193:40795)
0924 07:14:10.612748 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:10.612758 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:10.612809 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:10.612914 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:10.612914 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:10.612915 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 07:14:10.612962 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.612963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.613763 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.613764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.614345 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:10.614346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.614778 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.614779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:10.614818 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.614829 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 07:14:10.614973 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 07:14:10.614974 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 07:14:10.614999 (+ 25us) client_negotiation.cc:240] Negotiation successful
0924 07:14:10.615026 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":101,"threads_started":1}
I20260924 07:14:10.617362 12312 raft_consensus.cc:1261] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464 [term 1 FOLLOWER]: Refusing update from remote peer 4c0062cd8c25408b9c80fdfad9a795f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:10.617388 12172 raft_consensus.cc:1261] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 1 FOLLOWER]: Refusing update from remote peer 4c0062cd8c25408b9c80fdfad9a795f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:10.617555 12388 consensus_queue.cc:1035] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf898a555134660a839f74a62d13464" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:10.617630 12408 consensus_queue.cc:1035] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86c1a41d6d8c48a3b14d6362fb4a5fe7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:10.628604 11875 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:10.628684 11875 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:10.629241 11875 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a981328553c4f5ba60e7b8d7fc782c6: Sending DeleteTablet for 3 replicas of tablet 18aa6c8263f3471bbc0889926c63b938
I20260924 07:14:10.629467 12012 tablet_service.cc:1514] Processing DeleteTablet for tablet 18aa6c8263f3471bbc0889926c63b938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:10 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:49866
I20260924 07:14:10.629496 12152 tablet_service.cc:1514] Processing DeleteTablet for tablet 18aa6c8263f3471bbc0889926c63b938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:10 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:53750
I20260924 07:14:10.629515 12292 tablet_service.cc:1514] Processing DeleteTablet for tablet 18aa6c8263f3471bbc0889926c63b938 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:10 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:48626
I20260924 07:14:10.629556 12403 tablet_replica.cc:330] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0: stopping tablet replica
I20260924 07:14:10.629594 12402 tablet_replica.cc:330] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464: stopping tablet replica
I20260924 07:14:10.629613 12403 raft_consensus.cc:2229] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:10.629639 12402 raft_consensus.cc:2229] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:10.629669 12402 raft_consensus.cc:2258] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:10.629691 12403 raft_consensus.cc:2258] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:10.629757 12401 tablet_replica.cc:330] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: stopping tablet replica
I20260924 07:14:10.629802 12401 raft_consensus.cc:2229] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:10.629830 12401 raft_consensus.cc:2258] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:10.629956 12403 ts_tablet_manager.cc:1905] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:10.630155 12401 ts_tablet_manager.cc:1905] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:10.631646 12403 ts_tablet_manager.cc:1918] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:10.631651 12401 ts_tablet_manager.cc:1918] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:10.631695 12403 log.cc:1196] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-0/wal/wals/18aa6c8263f3471bbc0889926c63b938
I20260924 07:14:10.631697 12401 log.cc:1196] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/18aa6c8263f3471bbc0889926c63b938
I20260924 07:14:10.631902 12403 ts_tablet_manager.cc:1939] T 18aa6c8263f3471bbc0889926c63b938 P 4c0062cd8c25408b9c80fdfad9a795f0: Deleting consensus metadata
I20260924 07:14:10.631901 12401 ts_tablet_manager.cc:1939] T 18aa6c8263f3471bbc0889926c63b938 P 86c1a41d6d8c48a3b14d6362fb4a5fe7: Deleting consensus metadata
I20260924 07:14:10.632218 11861 catalog_manager.cc:4841] TS 4c0062cd8c25408b9c80fdfad9a795f0 (127.10.130.193:40945): tablet 18aa6c8263f3471bbc0889926c63b938 (table test-table [id=fd556f04154f4ec5b8bc9ef9332337a8]) successfully deleted
I20260924 07:14:10.632268 12402 ts_tablet_manager.cc:1905] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:10.632287 11861 catalog_manager.cc:4841] TS 86c1a41d6d8c48a3b14d6362fb4a5fe7 (127.10.130.194:43375): tablet 18aa6c8263f3471bbc0889926c63b938 (table test-table [id=fd556f04154f4ec5b8bc9ef9332337a8]) successfully deleted
I20260924 07:14:10.633350 12402 ts_tablet_manager.cc:1918] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:10.633394 12402 log.cc:1196] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/18aa6c8263f3471bbc0889926c63b938
I20260924 07:14:10.633558 12402 ts_tablet_manager.cc:1939] T 18aa6c8263f3471bbc0889926c63b938 P adf898a555134660a839f74a62d13464: Deleting consensus metadata
I20260924 07:14:10.634032 11861 catalog_manager.cc:4841] TS adf898a555134660a839f74a62d13464 (127.10.130.195:39829): tablet 18aa6c8263f3471bbc0889926c63b938 (table test-table [id=fd556f04154f4ec5b8bc9ef9332337a8]) successfully deleted
Sep 24 07:14:10 dist-test-slave-7q7m krb5kdc[11830](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234050, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.130.193@KRBTEST.COM
I20260924 07:14:10.644101 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.632369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49902 (local address 127.10.130.193:40945)
0924 07:14:10.632402 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.632404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.636154 (+ 3750us) server_negotiation.cc:407] Connection header received
0924 07:14:10.636212 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.636213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.636221 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.636264 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:10.636901 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.637340 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.638377 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.638552 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.638600 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.641564 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:10.641573 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:10.641573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:10.641581 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:10.643046 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.643408 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.643411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.643411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.643460 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.643630 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.643631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.643631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.643750 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:10.643844 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.643940 (+ 96us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.644037 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260924 07:14:10.644381 12072 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49902
I20260924 07:14:10.650998 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:10.645308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44758 (local address 127.10.130.254:33301)
0924 07:14:10.645349 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:10.645351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:10.645360 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:10.645404 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:10.645405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:10.645413 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:10.645454 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:10.645988 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.646453 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.647439 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:10.647611 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:10.647652 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:10.648878 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:10.648879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:10.648879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:10.648885 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:10.650050 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.650401 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.650401 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.650402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.650443 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:10.650625 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:10.650625 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:10.650625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:10.650711 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:10.650775 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:10.650888 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 07:14:10.650939 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 07:14:10.651360 11875 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44758
I20260924 07:14:10.652019 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 11947
I20260924 07:14:10.657493 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 12087
I20260924 07:14:10.662065 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 12227
I20260924 07:14:10.666124 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 11843
2026-09-24T07:14:10Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2004 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:10 dist-test-slave-7q7m krb5kdc[12438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:10 dist-test-slave-7q7m krb5kdc[12438](info): set up 2 sockets
Sep 24 07:14:10 dist-test-slave-7q7m krb5kdc[12438](info): commencing operation
krb5kdc: starting...
W20260924 07:14:11.623292 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.937s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:11 dist-test-slave-7q7m krb5kdc[12438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:14:11.728555 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:37223
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:37223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:11.805612 12451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:11.805800 12451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:11.805820 12451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:11.806922 12451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:11.806962 12451 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 07:14:11.806973 12451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:11.806985 12451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:11.806995 12451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:11.807020 12451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:11.808503 12451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:37223
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:37223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:11.808717 12451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:11.808919 12451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:11.813535 12457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:11.813617 12451 server_base.cc:1034] running on GCE node
W20260924 07:14:11.813544 12459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:11.813546 12456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:11.814045 12451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:11.814342 12451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:11.815482 12451 hybrid_clock.cc:648] HybridClock initialized: now 1790234051815473 us; error 45 us; skew 500 ppm
Sep 24 07:14:11 dist-test-slave-7q7m krb5kdc[12438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234051, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:11.817792 12451 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:14:11.818212 12451 webserver.cc:460] Webserver started at http://127.10.130.254:41399/ using document root <none> and password file <none>
I20260924 07:14:11.818397 12451 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:11.818480 12451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:11.818630 12451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:11.819635 12451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "d34275279e8e4780a03e7897ea52d589"
format_stamp: "Formatted at 2026-09-24 07:14:11 on dist-test-slave-7q7m"
server_key: "75e656fc3f1a989636f2607dabebf85e"
server_key_iv: "1381d61097b474404471bc606cc4d4fd"
server_key_version: "encryptionkey@0"
I20260924 07:14:11.820010 12451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "d34275279e8e4780a03e7897ea52d589"
format_stamp: "Formatted at 2026-09-24 07:14:11 on dist-test-slave-7q7m"
server_key: "75e656fc3f1a989636f2607dabebf85e"
server_key_iv: "1381d61097b474404471bc606cc4d4fd"
server_key_version: "encryptionkey@0"
I20260924 07:14:11.821214 12451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:11.821928 12466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:11.822117 12451 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:11.822188 12451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "d34275279e8e4780a03e7897ea52d589"
format_stamp: "Formatted at 2026-09-24 07:14:11 on dist-test-slave-7q7m"
server_key: "75e656fc3f1a989636f2607dabebf85e"
server_key_iv: "1381d61097b474404471bc606cc4d4fd"
server_key_version: "encryptionkey@0"
I20260924 07:14:11.822256 12451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:11.837821 12451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:11.839669 12451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:11.839787 12451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:11.844065 12451 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:37223
I20260924 07:14:11.844129 12528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:37223 every 8 connection(s)
I20260924 07:14:11.844425 12451 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:11.845031 12529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:11.847141 12529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589: Bootstrap starting.
I20260924 07:14:11.847680 12529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:11.847894 12529 log.cc:824] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:11.848524 12529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589: No bootstrap required, opened a new log
I20260924 07:14:11.850452 12529 raft_consensus.cc:348] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34275279e8e4780a03e7897ea52d589" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37223 } }
I20260924 07:14:11.850584 12529 raft_consensus.cc:374] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:11.850603 12529 raft_consensus.cc:729] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d34275279e8e4780a03e7897ea52d589, State: Initialized, Role: FOLLOWER
I20260924 07:14:11.850716 12529 consensus_queue.cc:260] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34275279e8e4780a03e7897ea52d589" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37223 } }
I20260924 07:14:11.850790 12529 raft_consensus.cc:388] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:11.850821 12529 raft_consensus.cc:482] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:11.850858 12529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:11.851302 12529 raft_consensus.cc:504] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34275279e8e4780a03e7897ea52d589" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37223 } }
I20260924 07:14:11.851382 12529 leader_election.cc:302] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d34275279e8e4780a03e7897ea52d589; no voters:
I20260924 07:14:11.851491 12529 leader_election.cc:288] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:11.851536 12532 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:11.851703 12532 raft_consensus.cc:686] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [term 1 LEADER]: Becoming Leader. State: Replica: d34275279e8e4780a03e7897ea52d589, State: Running, Role: LEADER
I20260924 07:14:11.851708 12529 sys_catalog.cc:564] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:11.851837 12532 consensus_queue.cc:237] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34275279e8e4780a03e7897ea52d589" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37223 } }
I20260924 07:14:11.852211 12534 sys_catalog.cc:455] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d34275279e8e4780a03e7897ea52d589. Latest consensus state: current_term: 1 leader_uuid: "d34275279e8e4780a03e7897ea52d589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34275279e8e4780a03e7897ea52d589" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37223 } } }
I20260924 07:14:11.852291 12534 sys_catalog.cc:458] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:11.852361 12533 sys_catalog.cc:455] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d34275279e8e4780a03e7897ea52d589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34275279e8e4780a03e7897ea52d589" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37223 } } }
I20260924 07:14:11.852473 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 12451
I20260924 07:14:11.852515 12533 sys_catalog.cc:458] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:11.852557 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:11.852730 10763 external_mini_cluster.cc:1428] Setting key 5fcc7cd61530b2bc1cd84a5781c1d274
I20260924 07:14:11.852769 12539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:11.853479 12539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:11.855022 12539 catalog_manager.cc:1347] Generated new cluster ID: 6d84e471041e4dcbbeb301a6abac8e27
I20260924 07:14:11.855069 12539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:11.859457 12539 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:11.859931 12539 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:11.874284 12539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589: Generated new TSK 0
I20260924 07:14:11.874475 12539 catalog_manager.cc:1514] Initializing in-progress tserver states...
Sep 24 07:14:11 dist-test-slave-7q7m krb5kdc[12438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234051, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:11.880941 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:11.853303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.10.130.254:37223)
0924 07:14:11.853450 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:11.853459 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:11.854188 (+ 729us) server_negotiation.cc:407] Connection header received
0924 07:14:11.854446 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:11.854454 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:11.854507 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:11.854617 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:11.872113 (+ 17496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:11.873814 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:11.874929 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:11.875348 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:11.875430 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:11.878016 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:11.878020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:11.878021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:11.878032 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:11.879504 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:11.879881 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:11.879883 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:11.879883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:11.879965 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:11.880178 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:11.880178 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:11.880179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:11.880383 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:11.880515 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:11.880611 (+ 96us) server_negotiation.cc:299] Negotiation successful
0924 07:14:11.880723 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:14:11.902283 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:37223
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:11.978873 12555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:11.979074 12555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:11.979096 12555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:11.980221 12555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:11.980259 12555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:11.980278 12555 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 07:14:11.980289 12555 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 07:14:11.980319 12555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:11.981735 12555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.130.254:37223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:11.981958 12555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:11.982162 12555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:11.986815 12563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:11.986821 12561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:11.986829 12560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:11.987145 12555 server_base.cc:1034] running on GCE node
I20260924 07:14:11.987339 12555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:11.987572 12555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:11.988708 12555 hybrid_clock.cc:648] HybridClock initialized: now 1790234051988685 us; error 35 us; skew 500 ppm
Sep 24 07:14:11 dist-test-slave-7q7m krb5kdc[12438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234051, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:11.991015 12555 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:14:11.991420 12555 webserver.cc:460] Webserver started at http://127.10.130.193:35773/ using document root <none> and password file <none>
I20260924 07:14:11.991616 12555 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:11.991664 12555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:11.991798 12555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:11.992812 12555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "7f62c1d5c71742f1bee8647a1a532d1d"
format_stamp: "Formatted at 2026-09-24 07:14:11 on dist-test-slave-7q7m"
server_key: "54df1a6e4d35aec882218add59b112c9"
server_key_iv: "0780fb5cee5414e55af57a1945dd75c0"
server_key_version: "encryptionkey@0"
I20260924 07:14:11.993088 12555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "7f62c1d5c71742f1bee8647a1a532d1d"
format_stamp: "Formatted at 2026-09-24 07:14:11 on dist-test-slave-7q7m"
server_key: "54df1a6e4d35aec882218add59b112c9"
server_key_iv: "0780fb5cee5414e55af57a1945dd75c0"
server_key_version: "encryptionkey@0"
I20260924 07:14:11.994201 12555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:11.994797 12570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:11.994964 12555 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:11.995033 12555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "7f62c1d5c71742f1bee8647a1a532d1d"
format_stamp: "Formatted at 2026-09-24 07:14:11 on dist-test-slave-7q7m"
server_key: "54df1a6e4d35aec882218add59b112c9"
server_key_iv: "0780fb5cee5414e55af57a1945dd75c0"
server_key_version: "encryptionkey@0"
I20260924 07:14:11.995100 12555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:12.015671 12555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:12.017831 12555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:12.017970 12555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:12.018462 12555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:12.018513 12555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:12.018538 12555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:12.018550 12555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 07:14:12 dist-test-slave-7q7m krb5kdc[12438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234051, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:12.025086 12555 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:41533
I20260924 07:14:12.025174 12685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:41533 every 8 connection(s)
I20260924 07:14:12.025503 12555 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:14:12.026532 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 12555
I20260924 07:14:12.026618 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:12.026775 10763 external_mini_cluster.cc:1428] Setting key 7ef53044671f84e2a80ba0f7739b38e3
I20260924 07:14:12.028486 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.019040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:50675 (local address 127.10.130.254:37223)
0924 07:14:12.019095 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.019098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.019132 (+ 34us) server_negotiation.cc:407] Connection header received
0924 07:14:12.019623 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.019624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.019637 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.019712 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:12.020508 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.021263 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.022278 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.022590 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.022662 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.025001 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:12.025002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:12.025002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:12.025010 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:12.026504 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.026889 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.026890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.026891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.026950 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.027215 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.027216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.027216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.027349 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:12.027417 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.028300 (+ 883us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.028395 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 07:14:12.028606 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.018866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:37223 (local address 127.10.130.193:50675)
0924 07:14:12.019021 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.019061 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.019527 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.019794 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.019799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.020124 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:12.020400 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.020406 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.021364 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.021365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.022208 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.022211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.022685 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.022686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.022826 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.023326 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:12.023335 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:12.024920 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:12.026583 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.026589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.026591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.026825 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.027022 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.027022 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.027023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.027133 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.028035 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:12.028042 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:12.028168 (+ 126us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.028260 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.028407 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260924 07:14:12.029022 12686 heartbeater.cc:347] Connected to a master server at 127.10.130.254:37223
I20260924 07:14:12.029112 12686 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:12.029278 12686 heartbeater.cc:511] Master 127.10.130.254:37223 requested a full tablet report, sending...
I20260924 07:14:12.029800 12482 ts_manager.cc:194] Registered new tserver with Master: 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193:41533)
I20260924 07:14:12.030546 12482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:50675
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
I20260924 07:14:12.035137 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.029980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:43269 (local address 127.10.130.254:37223)
0924 07:14:12.030011 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.030013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.030139 (+ 126us) server_negotiation.cc:407] Connection header received
0924 07:14:12.030164 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.030164 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.030173 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.030230 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:12.030573 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.031161 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.031861 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.032130 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.032186 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.032955 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:12.032956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:12.032957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:12.032963 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:12.034184 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.034470 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.034471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.034472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.034523 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.034691 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.034691 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.034692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.034793 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:12.034846 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.034985 (+ 139us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.035046 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260924 07:14:12.035149 12693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.029944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:37223 (local address 127.10.130.193:43269)
0924 07:14:12.030071 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.030080 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.030130 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.030254 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.030255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.030427 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:12.030498 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.030500 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.031261 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.031261 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.031782 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.031784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.032194 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.032195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.032260 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.032615 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:12.032620 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:12.032872 (+ 252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:12.034254 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.034254 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.034255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.034408 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.034580 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.034581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.034581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.034637 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.034849 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:12.034850 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:12.034894 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.034954 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.035049 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:14:12.050148 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:37223
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:12.127727 12695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:12.127942 12695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:12.128007 12695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:12.129199 12695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:12.129240 12695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:12.129266 12695 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 07:14:12.129280 12695 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 07:14:12.129303 12695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:12.130820 12695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.130.254:37223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:12.131062 12695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:12.131273 12695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:12.136113 12701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:12.136081 12703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:12.136081 12700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:12.136355 12695 server_base.cc:1034] running on GCE node
I20260924 07:14:12.136551 12695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:12.136809 12695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:12.137976 12695 hybrid_clock.cc:648] HybridClock initialized: now 1790234052137960 us; error 28 us; skew 500 ppm
Sep 24 07:14:12 dist-test-slave-7q7m krb5kdc[12438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234052, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:12.140424 12695 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:14:12.140873 12695 webserver.cc:460] Webserver started at http://127.10.130.194:40119/ using document root <none> and password file <none>
I20260924 07:14:12.141060 12695 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:12.141099 12695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:12.141196 12695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:12.142205 12695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "f89cecb7c4734778b5a32aded7233521"
format_stamp: "Formatted at 2026-09-24 07:14:12 on dist-test-slave-7q7m"
server_key: "8d46693d2c6d98262d67440587af0342"
server_key_iv: "dd1b38c3cb25af9682a589c1bc856897"
server_key_version: "encryptionkey@0"
I20260924 07:14:12.142494 12695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "f89cecb7c4734778b5a32aded7233521"
format_stamp: "Formatted at 2026-09-24 07:14:12 on dist-test-slave-7q7m"
server_key: "8d46693d2c6d98262d67440587af0342"
server_key_iv: "dd1b38c3cb25af9682a589c1bc856897"
server_key_version: "encryptionkey@0"
I20260924 07:14:12.143735 12695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 07:14:12.144479 12710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:12.144613 12695 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:12.144685 12695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "f89cecb7c4734778b5a32aded7233521"
format_stamp: "Formatted at 2026-09-24 07:14:12 on dist-test-slave-7q7m"
server_key: "8d46693d2c6d98262d67440587af0342"
server_key_iv: "dd1b38c3cb25af9682a589c1bc856897"
server_key_version: "encryptionkey@0"
I20260924 07:14:12.144739 12695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:12.159499 12695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:12.161226 12695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:12.161330 12695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:12.161789 12695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:12.161834 12695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:12.161862 12695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:12.161878 12695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 07:14:12 dist-test-slave-7q7m krb5kdc[12438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234052, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:12.168766 12695 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:44147
I20260924 07:14:12.169094 12695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:14:12.169091 12825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:44147 every 8 connection(s)
I20260924 07:14:12.171391 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.162185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:40129 (local address 127.10.130.254:37223)
0924 07:14:12.162252 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.162254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.162503 (+ 249us) server_negotiation.cc:407] Connection header received
0924 07:14:12.163148 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.163149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.163161 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.163208 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:12.164044 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.164657 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.166109 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.166326 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.166375 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.168810 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:12.168812 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:12.168812 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:12.168820 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:12.170154 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.170514 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.170515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.170516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.170572 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.170794 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.170795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.170795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.170907 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:12.170962 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.171229 (+ 267us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.171305 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 07:14:12.171533 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.162175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:37223 (local address 127.10.130.194:40129)
0924 07:14:12.162356 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.162404 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.163034 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.163297 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.163301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.163658 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:12.163943 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.163949 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.164727 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.164728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.166022 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.166025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.166408 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.166409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.166565 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.167079 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:12.167088 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:12.168713 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:12.170236 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.170242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.170244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.170442 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.170631 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.170631 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.170632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.170734 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.170966 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:12.170971 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:12.171099 (+ 128us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.171192 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.171335 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
I20260924 07:14:12.172019 12826 heartbeater.cc:347] Connected to a master server at 127.10.130.254:37223
I20260924 07:14:12.172094 12826 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:12.172276 12826 heartbeater.cc:511] Master 127.10.130.254:37223 requested a full tablet report, sending...
I20260924 07:14:12.172629 12483 ts_manager.cc:194] Registered new tserver with Master: f89cecb7c4734778b5a32aded7233521 (127.10.130.194:44147)
I20260924 07:14:12.173094 12483 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:40129
I20260924 07:14:12.174407 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 12695
I20260924 07:14:12.174482 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:12.174649 10763 external_mini_cluster.cc:1428] Setting key a76c43170647b20c074d6e2fad852968
I20260924 07:14:12.179503 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.173024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:55015 (local address 127.10.130.254:37223)
0924 07:14:12.173058 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.173059 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.173135 (+ 76us) server_negotiation.cc:407] Connection header received
0924 07:14:12.173204 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.173205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.173214 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.173259 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:12.173558 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.174023 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.174807 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.175051 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.175109 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.176034 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:12.176036 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:12.176036 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:12.176042 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:12.177938 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.178422 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.178423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.178423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.178482 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.178707 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.178709 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.178709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.178977 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:12.179044 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.179342 (+ 298us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.179416 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:14:12.179880 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.172937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:37223 (local address 127.10.130.194:55015)
0924 07:14:12.173078 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.173087 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.173148 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.173279 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.173280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.173433 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:12.173487 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.173488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.174120 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.174121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.174746 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.174749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.175108 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.175109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.175196 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.175573 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:12.175578 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:12.175915 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:12.178029 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.178030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.178030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.178345 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.178559 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.178560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.178560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.178624 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.179159 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:12.179159 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:12.179212 (+ 53us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.179681 (+ 469us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.179788 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":60,"threads_started":1}
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:14:12.198292 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:37223
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:45925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:12.276503 12835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:12.276712 12835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:12.276736 12835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:12.277855 12835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:12.277894 12835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:12.277913 12835 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 07:14:12.277925 12835 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 07:14:12.277953 12835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:12.279350 12835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:45925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.130.254:37223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:12.279563 12835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:12.279783 12835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:12.284431 12840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:12.284446 12843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:12.284431 12841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:12.284713 12835 server_base.cc:1034] running on GCE node
I20260924 07:14:12.284878 12835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:12.285110 12835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:12.286257 12835 hybrid_clock.cc:648] HybridClock initialized: now 1790234052286244 us; error 23 us; skew 500 ppm
Sep 24 07:14:12 dist-test-slave-7q7m krb5kdc[12438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234052, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:12.288599 12835 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:14:12.289028 12835 webserver.cc:460] Webserver started at http://127.10.130.195:36379/ using document root <none> and password file <none>
I20260924 07:14:12.289220 12835 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:12.289268 12835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:12.289393 12835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:12.290408 12835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "8753e90d9a07486ab5857e31c70d10e9"
format_stamp: "Formatted at 2026-09-24 07:14:12 on dist-test-slave-7q7m"
server_key: "a79c85227ab566a327be72551f0e6954"
server_key_iv: "40eaf2cbbc87e3c54ce22d9335062400"
server_key_version: "encryptionkey@0"
I20260924 07:14:12.290745 12835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "8753e90d9a07486ab5857e31c70d10e9"
format_stamp: "Formatted at 2026-09-24 07:14:12 on dist-test-slave-7q7m"
server_key: "a79c85227ab566a327be72551f0e6954"
server_key_iv: "40eaf2cbbc87e3c54ce22d9335062400"
server_key_version: "encryptionkey@0"
I20260924 07:14:12.291949 12835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:12.292651 12850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:12.292786 12835 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:12.292867 12835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "8753e90d9a07486ab5857e31c70d10e9"
format_stamp: "Formatted at 2026-09-24 07:14:12 on dist-test-slave-7q7m"
server_key: "a79c85227ab566a327be72551f0e6954"
server_key_iv: "40eaf2cbbc87e3c54ce22d9335062400"
server_key_version: "encryptionkey@0"
I20260924 07:14:12.292927 12835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:12.314374 12835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:12.316422 12835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:12.316562 12835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:12.317121 12835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:12.317163 12835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:12.317193 12835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:12.317220 12835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 07:14:12 dist-test-slave-7q7m krb5kdc[12438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234052, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:12.324127 12835 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:38587
I20260924 07:14:12.324200 12965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:38587 every 8 connection(s)
I20260924 07:14:12.324478 12835 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:12.327090 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.317553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:54199 (local address 127.10.130.254:37223)
0924 07:14:12.317615 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.317617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.317873 (+ 256us) server_negotiation.cc:407] Connection header received
0924 07:14:12.318512 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.318513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.318526 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.318594 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:12.319517 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.320258 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.321693 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.321956 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.322013 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.324394 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:12.324395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:12.324396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:12.324403 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:12.325752 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.326112 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.326114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.326114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.326172 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.326415 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.326416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.326416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.326553 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:12.326629 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.326926 (+ 297us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.327007 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 07:14:12.327301 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.317561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:37223 (local address 127.10.130.195:54199)
0924 07:14:12.317724 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.317774 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.318410 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.318679 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.318684 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.319035 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:12.319368 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.319376 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.320334 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.320334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.321602 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.321606 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.322033 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.322034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.322188 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.322659 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:12.322670 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:12.324312 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:12.325822 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.325828 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.325830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.326042 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.326229 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.326230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.326230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.326339 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.326624 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:12.326631 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:12.326788 (+ 157us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.326898 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.327063 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260924 07:14:12.327888 12966 heartbeater.cc:347] Connected to a master server at 127.10.130.254:37223
I20260924 07:14:12.327967 12966 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:12.328159 12966 heartbeater.cc:511] Master 127.10.130.254:37223 requested a full tablet report, sending...
I20260924 07:14:12.328512 12482 ts_manager.cc:194] Registered new tserver with Master: 8753e90d9a07486ab5857e31c70d10e9 (127.10.130.195:38587)
I20260924 07:14:12.328862 12482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:54199
I20260924 07:14:12.332624 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 12835
I20260924 07:14:12.332908 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:12.333112 10763 external_mini_cluster.cc:1428] Setting key 8db6af08509f4c890d94587f3524437e
I20260924 07:14:12.333963 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:14:12.334841 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.329503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:42469 (local address 127.10.130.254:37223)
0924 07:14:12.329547 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.329549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.329791 (+ 242us) server_negotiation.cc:407] Connection header received
0924 07:14:12.329854 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.329855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.329865 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.329905 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:12.330229 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.330695 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.331359 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.331548 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.331600 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.332469 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:12.332470 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:12.332471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:12.332476 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:12.333880 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.334232 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.334233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.334233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.334283 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.334455 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.334455 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.334455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.334538 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:12.334579 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.334730 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.334782 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 07:14:12.334884 12972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.329464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:37223 (local address 127.10.130.195:42469)
0924 07:14:12.329715 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.329743 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.329802 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.329907 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.329907 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.330091 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:12.330158 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.330160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.330785 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.330786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.331285 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.331286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.331609 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.331610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.331684 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.332127 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:12.332131 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:12.332385 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:12.333967 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.333968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.333968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.334167 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.334338 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.334339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.334339 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.334403 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.334592 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:12.334593 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:12.334639 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.334715 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.334803 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":59,"threads_started":1}
Sep 24 07:14:12 dist-test-slave-7q7m krb5kdc[12438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 07:14:12 dist-test-slave-7q7m krb5kdc[12438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:12.354918 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.348439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47640 (local address 127.10.130.254:37223)
0924 07:14:12.348509 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.348511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.348522 (+ 11us) server_negotiation.cc:407] Connection header received
0924 07:14:12.348547 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.348548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.348558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.348607 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:12.349093 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.349568 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.350259 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.350428 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.350471 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.352641 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:12.352642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:12.352643 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:12.352649 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:12.353928 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.354227 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.354228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.354229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.354281 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.354446 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.354447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.354447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.354551 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:12.354639 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.354757 (+ 118us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.354828 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 07:14:12.356462 12482 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:14:12.357928 12482 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:12.364148 12984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.359931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:44147 (local address 127.0.0.1:40482)
0924 07:14:12.360244 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.360279 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.360329 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.360768 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.360769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.360772 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:12.360819 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.360820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.361985 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.361986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.363037 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.363040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.363872 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.363873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.363930 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.363943 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.364044 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.364087 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":91,"thread_start_us":70,"threads_started":1}
I20260924 07:14:12.364199 12983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.360111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40482 (local address 127.10.130.194:44147)
0924 07:14:12.360533 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.360537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.360545 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:12.360579 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.360581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.360597 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.360689 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:12.360896 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.361930 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.363116 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.363795 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.363870 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.363925 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.364061 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.364098 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":64,"threads_started":1}
I20260924 07:14:12.364701 12980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.360141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:41533 (local address 127.0.0.1:60784)
0924 07:14:12.360764 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.360774 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.360848 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.361069 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.361069 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.361071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:12.361100 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.361101 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.362048 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.362049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.363280 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.363283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.364194 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.364195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.364536 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.364540 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.364598 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.364635 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":36,"thread_start_us":48,"threads_started":1}
I20260924 07:14:12.364770 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.360281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60784 (local address 127.10.130.193:41533)
0924 07:14:12.360458 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.360462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.360823 (+ 361us) server_negotiation.cc:407] Connection header received
0924 07:14:12.360934 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.360936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.360951 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.360999 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:12.361166 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.361962 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.363366 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.364107 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.364186 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.364238 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.364638 (+ 400us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.364676 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":74,"threads_started":1}
I20260924 07:14:12.364931 12760 tablet_service.cc:1467] Processing CreateTablet for tablet 9c209cde197b4551993418f185016a9c (DEFAULT_TABLE table=test-table [id=b214701148784217a8ee80f4164a7f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:12.365301 12981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.359931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38587 (local address 127.0.0.1:50236)
0924 07:14:12.360281 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.360291 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.360360 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.360543 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.360546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.360549 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:12.360617 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.360619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.361923 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.361924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.363280 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.363283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.365064 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.365065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.365153 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.365156 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.365201 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.365240 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":99,"thread_start_us":75,"threads_started":1}
I20260924 07:14:12.365307 12760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c209cde197b4551993418f185016a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:12.365517 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.360041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50236 (local address 127.10.130.195:38587)
0924 07:14:12.360230 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.360236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.360375 (+ 139us) server_negotiation.cc:407] Connection header received
0924 07:14:12.360435 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.360438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.360454 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.360511 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:12.360691 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.361842 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.364277 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.364919 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.364980 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.365033 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.365388 (+ 355us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.365431 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":87,"threads_started":1}
I20260924 07:14:12.365581 12620 tablet_service.cc:1467] Processing CreateTablet for tablet 9c209cde197b4551993418f185016a9c (DEFAULT_TABLE table=test-table [id=b214701148784217a8ee80f4164a7f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:12.365878 12620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c209cde197b4551993418f185016a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:12.366012 12900 tablet_service.cc:1467] Processing CreateTablet for tablet 9c209cde197b4551993418f185016a9c (DEFAULT_TABLE table=test-table [id=b214701148784217a8ee80f4164a7f24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:12.366252 12900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c209cde197b4551993418f185016a9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:12.368722 12986 tablet_bootstrap.cc:492] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9: Bootstrap starting.
I20260924 07:14:12.368891 12987 tablet_bootstrap.cc:492] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521: Bootstrap starting.
I20260924 07:14:12.369232 12988 tablet_bootstrap.cc:492] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d: Bootstrap starting.
I20260924 07:14:12.369495 12987 tablet_bootstrap.cc:654] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:12.369570 12986 tablet_bootstrap.cc:654] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:12.369808 12988 tablet_bootstrap.cc:654] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:12.369818 12987 log.cc:824] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:12.369859 12986 log.cc:824] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:12.370092 12988 log.cc:824] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:12.370539 12987 tablet_bootstrap.cc:492] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521: No bootstrap required, opened a new log
I20260924 07:14:12.370606 12987 ts_tablet_manager.cc:1397] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:12.370729 12986 tablet_bootstrap.cc:492] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9: No bootstrap required, opened a new log
I20260924 07:14:12.370774 12988 tablet_bootstrap.cc:492] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d: No bootstrap required, opened a new log
I20260924 07:14:12.370801 12986 ts_tablet_manager.cc:1397] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:12.370819 12988 ts_tablet_manager.cc:1397] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:12.372361 12987 raft_consensus.cc:348] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } }
I20260924 07:14:12.372491 12987 raft_consensus.cc:374] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:12.372520 12987 raft_consensus.cc:729] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89cecb7c4734778b5a32aded7233521, State: Initialized, Role: FOLLOWER
I20260924 07:14:12.372660 12987 consensus_queue.cc:260] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } }
I20260924 07:14:12.372742 12986 raft_consensus.cc:348] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } }
I20260924 07:14:12.372742 12988 raft_consensus.cc:348] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } }
I20260924 07:14:12.372900 12987 ts_tablet_manager.cc:1428] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:14:12.372900 12988 raft_consensus.cc:374] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:12.372900 12986 raft_consensus.cc:374] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:12.372936 12988 raft_consensus.cc:729] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62c1d5c71742f1bee8647a1a532d1d, State: Initialized, Role: FOLLOWER
I20260924 07:14:12.372982 12986 raft_consensus.cc:729] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8753e90d9a07486ab5857e31c70d10e9, State: Initialized, Role: FOLLOWER
I20260924 07:14:12.373006 12826 heartbeater.cc:503] Master 127.10.130.254:37223 was elected leader, sending a full tablet report...
I20260924 07:14:12.373102 12988 consensus_queue.cc:260] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } }
I20260924 07:14:12.373102 12986 consensus_queue.cc:260] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } }
I20260924 07:14:12.373346 12986 ts_tablet_manager.cc:1428] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:14:12.373374 12686 heartbeater.cc:503] Master 127.10.130.254:37223 was elected leader, sending a full tablet report...
I20260924 07:14:12.373564 12988 ts_tablet_manager.cc:1428] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:14:12.373642 12966 heartbeater.cc:503] Master 127.10.130.254:37223 was elected leader, sending a full tablet report...
W20260924 07:14:12.419674 12827 tablet.cc:2369] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:14:12.466563 12993 raft_consensus.cc:482] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:12.466682 12993 raft_consensus.cc:504] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } }
I20260924 07:14:12.466969 12993 leader_election.cc:288] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f89cecb7c4734778b5a32aded7233521 (127.10.130.194:44147), 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193:41533)
I20260924 07:14:12.469831 12983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.467261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:40351 (local address 127.10.130.194:44147)
0924 07:14:12.467323 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.467325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.467336 (+ 11us) server_negotiation.cc:407] Connection header received
0924 07:14:12.467365 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.467366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.467376 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.467444 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:12.467626 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.468179 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.469108 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.469483 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.469536 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.469544 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.469726 (+ 182us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.469762 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 07:14:12.469838 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.467394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:43015 (local address 127.10.130.193:41533)
0924 07:14:12.467434 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.467435 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.467514 (+ 79us) server_negotiation.cc:407] Connection header received
0924 07:14:12.467561 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.467562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.467570 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.467622 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:12.467762 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.468286 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.469164 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.469531 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.469597 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.469605 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.469741 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.469775 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:14:12.469839 12859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.467161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:44147 (local address 127.10.130.195:40351)
0924 07:14:12.467231 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.467243 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.467290 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.467476 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.467477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.467479 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:12.467546 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.467548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.468253 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.468254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.469011 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.469013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.469566 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.469567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.469654 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.469657 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.469697 (+ 40us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.469736 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260924 07:14:12.469854 12995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.467295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:41533 (local address 127.10.130.195:43015)
0924 07:14:12.467441 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.467452 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.467517 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.467633 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.467633 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.467635 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:12.467702 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.467704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.468363 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.468364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.469076 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.469078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.469613 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.469613 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.469669 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.469673 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.469722 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.469760 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":90,"threads_started":1}
I20260924 07:14:12.470209 12780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c209cde197b4551993418f185016a9c" candidate_uuid: "8753e90d9a07486ab5857e31c70d10e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f89cecb7c4734778b5a32aded7233521" is_pre_election: true
I20260924 07:14:12.470201 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c209cde197b4551993418f185016a9c" candidate_uuid: "8753e90d9a07486ab5857e31c70d10e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" is_pre_election: true
I20260924 07:14:12.470350 12780 raft_consensus.cc:2454] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8753e90d9a07486ab5857e31c70d10e9 in term 0.
I20260924 07:14:12.470350 12640 raft_consensus.cc:2454] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8753e90d9a07486ab5857e31c70d10e9 in term 0.
I20260924 07:14:12.470567 12852 leader_election.cc:302] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62c1d5c71742f1bee8647a1a532d1d, 8753e90d9a07486ab5857e31c70d10e9; no voters:
I20260924 07:14:12.470698 12993 raft_consensus.cc:2781] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:12.470765 12993 raft_consensus.cc:482] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:12.470782 12993 raft_consensus.cc:3037] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:12.471431 12993 raft_consensus.cc:504] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } }
I20260924 07:14:12.471535 12993 leader_election.cc:288] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [CANDIDATE]: Term 1 election: Requested vote from peers f89cecb7c4734778b5a32aded7233521 (127.10.130.194:44147), 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193:41533)
I20260924 07:14:12.471696 12780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c209cde197b4551993418f185016a9c" candidate_uuid: "8753e90d9a07486ab5857e31c70d10e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f89cecb7c4734778b5a32aded7233521"
I20260924 07:14:12.471757 12780 raft_consensus.cc:3037] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:12.471725 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c209cde197b4551993418f185016a9c" candidate_uuid: "8753e90d9a07486ab5857e31c70d10e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62c1d5c71742f1bee8647a1a532d1d"
I20260924 07:14:12.471807 12640 raft_consensus.cc:3037] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:12.472456 12780 raft_consensus.cc:2454] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8753e90d9a07486ab5857e31c70d10e9 in term 1.
I20260924 07:14:12.472601 12640 raft_consensus.cc:2454] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8753e90d9a07486ab5857e31c70d10e9 in term 1.
I20260924 07:14:12.472610 12852 leader_election.cc:302] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8753e90d9a07486ab5857e31c70d10e9, f89cecb7c4734778b5a32aded7233521; no voters:
I20260924 07:14:12.472702 12993 raft_consensus.cc:2781] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:12.472847 12993 raft_consensus.cc:686] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [term 1 LEADER]: Becoming Leader. State: Replica: 8753e90d9a07486ab5857e31c70d10e9, State: Running, Role: LEADER
I20260924 07:14:12.472929 12993 consensus_queue.cc:237] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } }
I20260924 07:14:12.473752 12482 catalog_manager.cc:5495] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8753e90d9a07486ab5857e31c70d10e9 (127.10.130.195). New cstate: current_term: 1 leader_uuid: "8753e90d9a07486ab5857e31c70d10e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } health_report { overall_health: UNKNOWN } } }
Sep 24 07:14:12 dist-test-slave-7q7m krb5kdc[12438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234051, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
W20260924 07:14:12.526229 12687 tablet.cc:2369] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:14:12.526978 13003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.520604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:37223 (local address 127.0.0.1:47652)
0924 07:14:12.520752 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.520764 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.520827 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.520943 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.520944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.521257 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:12.521302 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.521305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.521926 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.521927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.522441 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.522442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.522768 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.522769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.522816 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.523342 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:12.523347 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:12.524691 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:12.525916 (+ 1225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.525922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.525924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.526143 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.526346 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:12.526347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:12.526347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:12.526411 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:12.526649 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:12.526656 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:12.526701 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.526755 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.526868 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260924 07:14:12.527030 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.520662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47652 (local address 127.10.130.254:37223)
0924 07:14:12.520725 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.520727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.520821 (+ 94us) server_negotiation.cc:407] Connection header received
0924 07:14:12.520874 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.520874 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.520884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.520931 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:12.521383 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.521853 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.522536 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.522707 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.522759 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.524776 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:12.524777 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:12.524777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:12.524784 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:12.525850 (+ 1066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.526217 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.526218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.526218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.526281 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:12.526475 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:12.526476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:12.526477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:12.526589 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:12.526646 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.526786 (+ 140us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.526928 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":30}
I20260924 07:14:12.528970 12482 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:47652:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 07:14:12.529191 12482 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:12.530556 12620 tablet_service.cc:1467] Processing CreateTablet for tablet 7ac78d428be140aaa289634ef5dd2c8e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8372936bacf348f0a5afe092ccb10ea9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 07:14:12.530586 12760 tablet_service.cc:1467] Processing CreateTablet for tablet 7ac78d428be140aaa289634ef5dd2c8e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8372936bacf348f0a5afe092ccb10ea9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 07:14:12.530728 12760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ac78d428be140aaa289634ef5dd2c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:12.530728 12620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ac78d428be140aaa289634ef5dd2c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:12.532146 12987 tablet_bootstrap.cc:492] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521: Bootstrap starting.
I20260924 07:14:12.532155 12988 tablet_bootstrap.cc:492] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d: Bootstrap starting.
I20260924 07:14:12.532253 12900 tablet_service.cc:1467] Processing CreateTablet for tablet 7ac78d428be140aaa289634ef5dd2c8e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8372936bacf348f0a5afe092ccb10ea9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 07:14:12.532377 12900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ac78d428be140aaa289634ef5dd2c8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:12.532583 12988 tablet_bootstrap.cc:654] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:12.532817 12987 tablet_bootstrap.cc:654] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:12.533397 12987 tablet_bootstrap.cc:492] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521: No bootstrap required, opened a new log
I20260924 07:14:12.533437 12987 ts_tablet_manager.cc:1397] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:12.533581 12987 raft_consensus.cc:348] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.533646 12987 raft_consensus.cc:374] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:12.533660 12987 raft_consensus.cc:729] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89cecb7c4734778b5a32aded7233521, State: Initialized, Role: FOLLOWER
I20260924 07:14:12.533711 12987 consensus_queue.cc:260] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.533792 12987 ts_tablet_manager.cc:1428] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:12.533864 12992 tablet_replica.cc:439] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } }
I20260924 07:14:12.533898 12986 tablet_bootstrap.cc:492] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9: Bootstrap starting.
I20260924 07:14:12.533932 12992 tablet_replica.cc:442] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:12.534298 12986 tablet_bootstrap.cc:654] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:12.534349 12988 tablet_bootstrap.cc:492] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d: No bootstrap required, opened a new log
I20260924 07:14:12.534399 12988 ts_tablet_manager.cc:1397] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:12.534534 12988 raft_consensus.cc:348] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.534587 12988 raft_consensus.cc:374] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:12.534605 12988 raft_consensus.cc:729] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62c1d5c71742f1bee8647a1a532d1d, State: Initialized, Role: FOLLOWER
I20260924 07:14:12.534653 12988 consensus_queue.cc:260] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.534736 12988 ts_tablet_manager.cc:1428] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:12.534885 12986 tablet_bootstrap.cc:492] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9: No bootstrap required, opened a new log
I20260924 07:14:12.534924 12986 ts_tablet_manager.cc:1397] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:12.534927 12994 tablet_replica.cc:439] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } }
I20260924 07:14:12.534994 12994 tablet_replica.cc:442] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:12.535063 12986 raft_consensus.cc:348] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.535135 12986 raft_consensus.cc:374] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:12.535156 12986 raft_consensus.cc:729] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8753e90d9a07486ab5857e31c70d10e9, State: Initialized, Role: FOLLOWER
I20260924 07:14:12.535192 12986 consensus_queue.cc:260] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.535277 12986 ts_tablet_manager.cc:1428] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:12.535310 12993 tablet_replica.cc:439] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } }
I20260924 07:14:12.535367 12993 tablet_replica.cc:442] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9: This TxnStatusTablet replica's current role is: FOLLOWER
W20260924 07:14:12.575008 12967 tablet.cc:2369] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:14:12.608775 12992 raft_consensus.cc:482] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:12.608860 12992 raft_consensus.cc:504] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.609193 12992 leader_election.cc:288] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193:41533), 8753e90d9a07486ab5857e31c70d10e9 (127.10.130.195:38587)
I20260924 07:14:12.611914 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.609430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:43251 (local address 127.10.130.193:41533)
0924 07:14:12.609523 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.609525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.609534 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:12.609562 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.609563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.609573 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.609625 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:12.609781 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.610273 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.611202 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.611564 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.611618 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.611626 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.611803 (+ 177us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.611843 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 07:14:12.611909 12719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.609348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:41533 (local address 127.10.130.194:43251)
0924 07:14:12.609408 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.609421 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.609473 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.609636 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.609637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.609639 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:12.609711 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.609713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.610349 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.610350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.611121 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.611123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.611645 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.611646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.611734 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.611738 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.611779 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.611809 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":19}
I20260924 07:14:12.611935 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.609484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:57123 (local address 127.10.130.195:38587)
0924 07:14:12.609524 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.609526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.609583 (+ 57us) server_negotiation.cc:407] Connection header received
0924 07:14:12.609641 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.609641 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.609650 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.609705 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:12.609855 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.610299 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.611254 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.611606 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.611683 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.611692 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.611842 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.611881 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:14:12.612035 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.609348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38587 (local address 127.10.130.194:57123)
0924 07:14:12.609510 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.609521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.609584 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.609718 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.609718 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.609720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:12.609793 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.609795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.610384 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.610384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.611163 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.611166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.611717 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.611718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.611769 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.611773 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.611824 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.611858 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":13,"thread_start_us":87,"threads_started":1}
I20260924 07:14:12.612221 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac78d428be140aaa289634ef5dd2c8e" candidate_uuid: "f89cecb7c4734778b5a32aded7233521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" is_pre_election: true
I20260924 07:14:12.612310 12640 raft_consensus.cc:2454] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f89cecb7c4734778b5a32aded7233521 in term 0.
I20260924 07:14:12.612355 12920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac78d428be140aaa289634ef5dd2c8e" candidate_uuid: "f89cecb7c4734778b5a32aded7233521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8753e90d9a07486ab5857e31c70d10e9" is_pre_election: true
I20260924 07:14:12.612469 12920 raft_consensus.cc:2454] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f89cecb7c4734778b5a32aded7233521 in term 0.
I20260924 07:14:12.612498 12712 leader_election.cc:302] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62c1d5c71742f1bee8647a1a532d1d, f89cecb7c4734778b5a32aded7233521; no voters:
I20260924 07:14:12.612617 12992 raft_consensus.cc:2781] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:12.612664 12992 raft_consensus.cc:482] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:12.612684 12992 raft_consensus.cc:3037] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:12.613333 12992 raft_consensus.cc:504] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.613451 12992 leader_election.cc:288] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193:41533), 8753e90d9a07486ab5857e31c70d10e9 (127.10.130.195:38587)
I20260924 07:14:12.613595 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac78d428be140aaa289634ef5dd2c8e" candidate_uuid: "f89cecb7c4734778b5a32aded7233521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62c1d5c71742f1bee8647a1a532d1d"
I20260924 07:14:12.613619 12920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ac78d428be140aaa289634ef5dd2c8e" candidate_uuid: "f89cecb7c4734778b5a32aded7233521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8753e90d9a07486ab5857e31c70d10e9"
I20260924 07:14:12.613659 12640 raft_consensus.cc:3037] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:12.613674 12920 raft_consensus.cc:3037] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:12.614208 12640 raft_consensus.cc:2454] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f89cecb7c4734778b5a32aded7233521 in term 1.
I20260924 07:14:12.614249 12920 raft_consensus.cc:2454] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f89cecb7c4734778b5a32aded7233521 in term 1.
I20260924 07:14:12.614379 12713 leader_election.cc:302] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8753e90d9a07486ab5857e31c70d10e9, f89cecb7c4734778b5a32aded7233521; no voters:
I20260924 07:14:12.614463 12992 raft_consensus.cc:2781] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:12.614585 12992 raft_consensus.cc:686] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [term 1 LEADER]: Becoming Leader. State: Replica: f89cecb7c4734778b5a32aded7233521, State: Running, Role: LEADER
I20260924 07:14:12.614662 12992 consensus_queue.cc:237] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.615062 13008 tablet_replica.cc:439] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521: TxnStatusTablet state changed. Reason: New leader f89cecb7c4734778b5a32aded7233521. Latest consensus state: current_term: 1 leader_uuid: "f89cecb7c4734778b5a32aded7233521" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } }
I20260924 07:14:12.615146 13008 tablet_replica.cc:442] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521: This TxnStatusTablet replica's current role is: LEADER
I20260924 07:14:12.615363 13010 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 07:14:12.615473 13010 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 07:14:12.615393 12483 catalog_manager.cc:5495] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 reported cstate change: term changed from 0 to 1, leader changed from <none> to f89cecb7c4734778b5a32aded7233521 (127.10.130.194). New cstate: current_term: 1 leader_uuid: "f89cecb7c4734778b5a32aded7233521" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } health_report { overall_health: UNKNOWN } } }
I20260924 07:14:12.626184 12983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.623477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40484 (local address 127.10.130.194:44147)
0924 07:14:12.623522 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.623524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.623576 (+ 52us) server_negotiation.cc:407] Connection header received
0924 07:14:12.623653 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.623654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.623668 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.623720 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:12.623828 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.624391 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.625454 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.625666 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.625714 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.625890 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:12.625978 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:12.626047 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.626100 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.626122 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 07:14:12.626223 13003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.623436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:44147 (local address 127.0.0.1:40484)
0924 07:14:12.623525 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.623535 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.623587 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.623726 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.623727 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.623728 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 07:14:12.623775 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.623777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.624464 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.624465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.625379 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.625381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.625733 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.625734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.625798 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.625815 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 07:14:12.626042 (+ 227us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 07:14:12.626047 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.626097 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.626143 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260924 07:14:12.627887 12640 raft_consensus.cc:1261] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Refusing update from remote peer f89cecb7c4734778b5a32aded7233521: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:12.627888 12920 raft_consensus.cc:1261] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Refusing update from remote peer f89cecb7c4734778b5a32aded7233521: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:12.628182 13008 consensus_queue.cc:1035] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:12.628304 12992 consensus_queue.cc:1035] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:12.629431 12994 tablet_replica.cc:439] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d: TxnStatusTablet state changed. Reason: New leader f89cecb7c4734778b5a32aded7233521. Latest consensus state: current_term: 1 leader_uuid: "f89cecb7c4734778b5a32aded7233521" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } }
I20260924 07:14:12.629523 12994 tablet_replica.cc:442] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:12.629601 12993 tablet_replica.cc:439] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9: TxnStatusTablet state changed. Reason: New leader f89cecb7c4734778b5a32aded7233521. Latest consensus state: current_term: 1 leader_uuid: "f89cecb7c4734778b5a32aded7233521" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } }
I20260924 07:14:12.629673 12993 tablet_replica.cc:442] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:12.630162 12992 tablet_replica.cc:439] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f89cecb7c4734778b5a32aded7233521" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } }
I20260924 07:14:12.630236 12992 tablet_replica.cc:442] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521: This TxnStatusTablet replica's current role is: LEADER
I20260924 07:14:12.630319 13013 mvcc.cc:205] Tried to move back new op lower bound from 7332798679561609216 to 7332798679510065152. Current Snapshot: MvccSnapshot[applied={T|T < 7332798679561609216}]
I20260924 07:14:12.630625 12993 tablet_replica.cc:439] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f89cecb7c4734778b5a32aded7233521" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } }
I20260924 07:14:12.630692 12993 tablet_replica.cc:442] T 7ac78d428be140aaa289634ef5dd2c8e P 8753e90d9a07486ab5857e31c70d10e9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:12.630756 13015 mvcc.cc:205] Tried to move back new op lower bound from 7332798679561609216 to 7332798679510065152. Current Snapshot: MvccSnapshot[applied={T|T < 7332798679561609216}]
I20260924 07:14:12.630797 12994 tablet_replica.cc:439] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f89cecb7c4734778b5a32aded7233521" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } }
I20260924 07:14:12.630874 12994 tablet_replica.cc:442] T 7ac78d428be140aaa289634ef5dd2c8e P 7f62c1d5c71742f1bee8647a1a532d1d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:12.632057 12992 tablet_replica.cc:439] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f89cecb7c4734778b5a32aded7233521" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } } }
I20260924 07:14:12.632129 12992 tablet_replica.cc:442] T 7ac78d428be140aaa289634ef5dd2c8e P f89cecb7c4734778b5a32aded7233521: This TxnStatusTablet replica's current role is: LEADER
I20260924 07:14:12.632233 13014 mvcc.cc:205] Tried to move back new op lower bound from 7332798679561609216 to 7332798679510065152. Current Snapshot: MvccSnapshot[applied={T|T < 7332798679561609216}]
I20260924 07:14:12.635305 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.633108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50244 (local address 127.10.130.195:38587)
0924 07:14:12.633165 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.633166 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.633173 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:12.633199 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.633200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.633214 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.633269 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:12.633399 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.633852 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.634596 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.634797 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.634848 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.635004 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:12.635102 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:12.635166 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.635221 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.635241 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 07:14:12.640221 12983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.637822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:34803 (local address 127.10.130.194:44147)
0924 07:14:12.637875 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.637877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.637883 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:14:12.637900 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.637900 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.637908 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.637949 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:12.638094 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.638516 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.639586 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.639754 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.639797 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.640017 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:12.640045 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:12.640090 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.640139 (+ 49us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.640157 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 07:14:12.640246 12972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.637626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:44147 (local address 127.10.130.195:34803)
0924 07:14:12.637699 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.637710 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.637752 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.637977 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.637977 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.637979 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 07:14:12.638033 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.638035 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.638583 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.638584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.639515 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.639517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.639819 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.639820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.639885 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.639903 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 07:14:12.640091 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 07:14:12.640092 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.640133 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.640174 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 07:14:12.642714 12780 raft_consensus.cc:1261] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Refusing update from remote peer 8753e90d9a07486ab5857e31c70d10e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:12.642872 12640 raft_consensus.cc:1261] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Refusing update from remote peer 8753e90d9a07486ab5857e31c70d10e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:12.642970 12993 consensus_queue.cc:1035] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:12.643105 12996 consensus_queue.cc:1035] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:12.651597 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.648941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:48893 (local address 127.10.130.195:38587)
0924 07:14:12.648977 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:12.648979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:12.648988 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:12.649047 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:12.649048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:12.649057 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:12.649104 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:12.649235 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.649751 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.650854 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.651142 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.651207 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.651397 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:12.651423 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:12.651459 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:12.651515 (+ 56us) server_negotiation.cc:299] Negotiation successful
0924 07:14:12.651530 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:14:12.651624 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:12.648886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38587 (local address 127.10.130.194:48893)
0924 07:14:12.648955 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:12.648966 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:12.649002 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:12.649109 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:12.649109 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:12.649111 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 07:14:12.649169 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.649171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.649824 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.649825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.650789 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:12.650792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:12.651233 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:12.651233 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:12.651302 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:12.651317 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 07:14:12.651460 (+ 143us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 07:14:12.651462 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:14:12.651513 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 07:14:12.651550 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 07:14:12.657927 12481 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:12.658023 12481 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:12.658728 12481 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589: Sending DeleteTablet for 3 replicas of tablet 9c209cde197b4551993418f185016a9c
I20260924 07:14:12.659034 12760 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c209cde197b4551993418f185016a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:12 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:40482
I20260924 07:14:12.659034 12900 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c209cde197b4551993418f185016a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:12 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:50236
I20260924 07:14:12.659333 12620 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c209cde197b4551993418f185016a9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:12 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:60784
I20260924 07:14:12.659543 13033 tablet_replica.cc:330] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d: stopping tablet replica
I20260924 07:14:12.659691 13033 raft_consensus.cc:2229] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:12.659806 13033 raft_consensus.cc:2258] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:12.659765 12481 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:14:12.659897 12481 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:12.660992 12760 tablet_service.cc:1467] Processing CreateTablet for tablet 9ed93846d94f4e6d9fd3a6d9be3459a3 (DEFAULT_TABLE table=test-table [id=c8e3c89964284f2790c12b8a52143c01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:12.661046 12620 tablet_service.cc:1467] Processing CreateTablet for tablet 9ed93846d94f4e6d9fd3a6d9be3459a3 (DEFAULT_TABLE table=test-table [id=c8e3c89964284f2790c12b8a52143c01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:12.661116 12760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ed93846d94f4e6d9fd3a6d9be3459a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:12.661139 12620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ed93846d94f4e6d9fd3a6d9be3459a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:12.662370 12988 tablet_bootstrap.cc:492] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d: Bootstrap starting.
I20260924 07:14:12.662367 12987 tablet_bootstrap.cc:492] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521: Bootstrap starting.
I20260924 07:14:12.662736 12988 tablet_bootstrap.cc:654] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:12.662812 12987 tablet_bootstrap.cc:654] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:12.663059 12900 tablet_service.cc:1467] Processing CreateTablet for tablet 9ed93846d94f4e6d9fd3a6d9be3459a3 (DEFAULT_TABLE table=test-table [id=c8e3c89964284f2790c12b8a52143c01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:12.663172 12900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ed93846d94f4e6d9fd3a6d9be3459a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:12.664357 12988 tablet_bootstrap.cc:492] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d: No bootstrap required, opened a new log
I20260924 07:14:12.664398 12988 ts_tablet_manager.cc:1397] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:12.664528 12988 raft_consensus.cc:348] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.664583 12988 raft_consensus.cc:374] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:12.664600 12988 raft_consensus.cc:729] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62c1d5c71742f1bee8647a1a532d1d, State: Initialized, Role: FOLLOWER
I20260924 07:14:12.664636 12988 consensus_queue.cc:260] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.664712 12988 ts_tablet_manager.cc:1428] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:12.664963 12986 tablet_bootstrap.cc:492] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9: Bootstrap starting.
I20260924 07:14:12.665354 12986 tablet_bootstrap.cc:654] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:12.665504 13033 ts_tablet_manager.cc:1905] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:12.666796 13031 tablet_replica.cc:330] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9: stopping tablet replica
I20260924 07:14:12.666903 13031 raft_consensus.cc:2229] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:12.667008 13031 raft_consensus.cc:2258] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:12.667169 13033 ts_tablet_manager.cc:1918] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:12.667253 13033 log.cc:1196] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/wal/wals/9c209cde197b4551993418f185016a9c
I20260924 07:14:12.667485 13033 ts_tablet_manager.cc:1939] T 9c209cde197b4551993418f185016a9c P 7f62c1d5c71742f1bee8647a1a532d1d: Deleting consensus metadata
I20260924 07:14:12.667590 13031 ts_tablet_manager.cc:1905] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:12.667856 12468 catalog_manager.cc:4841] TS 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193:41533): tablet 9c209cde197b4551993418f185016a9c (table test-table [id=b214701148784217a8ee80f4164a7f24]) successfully deleted
I20260924 07:14:12.668051 13032 tablet_replica.cc:330] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521: stopping tablet replica
I20260924 07:14:12.668169 13032 raft_consensus.cc:2229] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:12.668237 13032 raft_consensus.cc:2258] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:12.668627 13032 ts_tablet_manager.cc:1905] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:12.669687 13031 ts_tablet_manager.cc:1918] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:12.669799 13031 log.cc:1196] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/9c209cde197b4551993418f185016a9c
I20260924 07:14:12.670104 13032 ts_tablet_manager.cc:1918] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:12.670179 13032 log.cc:1196] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/9c209cde197b4551993418f185016a9c
I20260924 07:14:12.670289 13031 ts_tablet_manager.cc:1939] T 9c209cde197b4551993418f185016a9c P 8753e90d9a07486ab5857e31c70d10e9: Deleting consensus metadata
I20260924 07:14:12.670396 13032 ts_tablet_manager.cc:1939] T 9c209cde197b4551993418f185016a9c P f89cecb7c4734778b5a32aded7233521: Deleting consensus metadata
I20260924 07:14:12.670727 12468 catalog_manager.cc:4841] TS f89cecb7c4734778b5a32aded7233521 (127.10.130.194:44147): tablet 9c209cde197b4551993418f185016a9c (table test-table [id=b214701148784217a8ee80f4164a7f24]) successfully deleted
I20260924 07:14:12.671028 12469 catalog_manager.cc:4841] TS 8753e90d9a07486ab5857e31c70d10e9 (127.10.130.195:38587): tablet 9c209cde197b4551993418f185016a9c (table test-table [id=b214701148784217a8ee80f4164a7f24]) successfully deleted
I20260924 07:14:12.672346 12987 tablet_bootstrap.cc:492] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521: No bootstrap required, opened a new log
I20260924 07:14:12.672391 12987 ts_tablet_manager.cc:1397] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260924 07:14:12.672540 12987 raft_consensus.cc:348] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.672602 12987 raft_consensus.cc:374] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:12.672619 12987 raft_consensus.cc:729] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89cecb7c4734778b5a32aded7233521, State: Initialized, Role: FOLLOWER
I20260924 07:14:12.672659 12987 consensus_queue.cc:260] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.672742 12987 ts_tablet_manager.cc:1428] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:12.680374 12986 tablet_bootstrap.cc:492] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9: No bootstrap required, opened a new log
I20260924 07:14:12.680430 12986 ts_tablet_manager.cc:1397] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20260924 07:14:12.680573 12986 raft_consensus.cc:348] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.680644 12986 raft_consensus.cc:374] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:12.680661 12986 raft_consensus.cc:729] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8753e90d9a07486ab5857e31c70d10e9, State: Initialized, Role: FOLLOWER
I20260924 07:14:12.680701 12986 consensus_queue.cc:260] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:12.680795 12986 ts_tablet_manager.cc:1428] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:13.040819 12994 raft_consensus.cc:482] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:13.040910 12994 raft_consensus.cc:504] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:13.041230 12994 leader_election.cc:288] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f89cecb7c4734778b5a32aded7233521 (127.10.130.194:44147), 8753e90d9a07486ab5857e31c70d10e9 (127.10.130.195:38587)
I20260924 07:14:13.043885 12982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:13.041560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:52647 (local address 127.10.130.195:38587)
0924 07:14:13.041610 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:13.041612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:13.041632 (+ 20us) server_negotiation.cc:407] Connection header received
0924 07:14:13.041667 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:13.041668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:13.041680 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:13.041728 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:13.041867 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.042329 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.043251 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.043595 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.043653 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:13.043663 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:13.043794 (+ 131us) server_negotiation.cc:299] Negotiation successful
0924 07:14:13.043827 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:14:13.043942 12983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:13.041485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:55035 (local address 127.10.130.194:44147)
0924 07:14:13.041536 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:13.041539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:13.041760 (+ 221us) server_negotiation.cc:407] Connection header received
0924 07:14:13.041781 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:13.041782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:13.041788 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:13.041825 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:13.041952 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.042410 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.043286 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.043637 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.043695 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:13.043703 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:13.043852 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 07:14:13.043881 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:14:13.043892 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:13.041423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38587 (local address 127.10.130.193:52647)
0924 07:14:13.041546 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:13.041558 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:13.041611 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:13.041740 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:13.041740 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:13.041742 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:13.041795 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:13.041796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.042404 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.042405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:13.043163 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:13.043165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.043679 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.043680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:13.043727 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:13.043730 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:13.043774 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 07:14:13.043813 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":57,"threads_started":1}
I20260924 07:14:13.044003 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:13.041407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:44147 (local address 127.10.130.193:55035)
0924 07:14:13.041529 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:13.041539 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:13.041605 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:13.041846 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:13.041846 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:13.041849 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:13.041895 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:13.041896 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.042505 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.042506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:13.043203 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:13.043205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.043713 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.043714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:13.043789 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:13.043792 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:13.043839 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 07:14:13.043875 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":56,"threads_started":1}
I20260924 07:14:13.044293 12920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ed93846d94f4e6d9fd3a6d9be3459a3" candidate_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8753e90d9a07486ab5857e31c70d10e9" is_pre_election: true
I20260924 07:14:13.044302 12780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ed93846d94f4e6d9fd3a6d9be3459a3" candidate_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f89cecb7c4734778b5a32aded7233521" is_pre_election: true
I20260924 07:14:13.044375 12920 raft_consensus.cc:2454] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62c1d5c71742f1bee8647a1a532d1d in term 0.
I20260924 07:14:13.044384 12780 raft_consensus.cc:2454] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f62c1d5c71742f1bee8647a1a532d1d in term 0.
I20260924 07:14:13.044528 12572 leader_election.cc:302] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62c1d5c71742f1bee8647a1a532d1d, f89cecb7c4734778b5a32aded7233521; no voters:
I20260924 07:14:13.044674 12994 raft_consensus.cc:2781] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:13.044723 12994 raft_consensus.cc:482] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:13.044747 12994 raft_consensus.cc:3037] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:13.045502 12994 raft_consensus.cc:504] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:13.045614 12994 leader_election.cc:288] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [CANDIDATE]: Term 1 election: Requested vote from peers f89cecb7c4734778b5a32aded7233521 (127.10.130.194:44147), 8753e90d9a07486ab5857e31c70d10e9 (127.10.130.195:38587)
I20260924 07:14:13.045754 12780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ed93846d94f4e6d9fd3a6d9be3459a3" candidate_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f89cecb7c4734778b5a32aded7233521"
I20260924 07:14:13.045758 12920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ed93846d94f4e6d9fd3a6d9be3459a3" candidate_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8753e90d9a07486ab5857e31c70d10e9"
I20260924 07:14:13.045825 12780 raft_consensus.cc:3037] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:13.045825 12920 raft_consensus.cc:3037] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:13.046375 12780 raft_consensus.cc:2454] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62c1d5c71742f1bee8647a1a532d1d in term 1.
I20260924 07:14:13.046394 12920 raft_consensus.cc:2454] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f62c1d5c71742f1bee8647a1a532d1d in term 1.
I20260924 07:14:13.046526 12572 leader_election.cc:302] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62c1d5c71742f1bee8647a1a532d1d, f89cecb7c4734778b5a32aded7233521; no voters:
I20260924 07:14:13.046615 12994 raft_consensus.cc:2781] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:13.046742 12994 raft_consensus.cc:686] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 LEADER]: Becoming Leader. State: Replica: 7f62c1d5c71742f1bee8647a1a532d1d, State: Running, Role: LEADER
I20260924 07:14:13.046805 12994 consensus_queue.cc:237] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:13.047536 12483 catalog_manager.cc:5495] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193). New cstate: current_term: 1 leader_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } health_report { overall_health: UNKNOWN } } }
I20260924 07:14:13.108790 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:13.106528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60788 (local address 127.10.130.193:41533)
0924 07:14:13.106573 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:13.106575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:13.106585 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:14:13.106644 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:13.106644 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:13.106663 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:13.106726 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:13.106817 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.107347 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.108065 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.108256 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.108314 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:13.108511 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:13.108619 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:13.108683 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:13.108713 (+ 30us) server_negotiation.cc:299] Negotiation successful
0924 07:14:13.108732 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 07:14:13.113384 12983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:13.111259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:58353 (local address 127.10.130.194:44147)
0924 07:14:13.111316 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:13.111318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:13.111370 (+ 52us) server_negotiation.cc:407] Connection header received
0924 07:14:13.111408 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:13.111408 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:13.111417 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:13.111457 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:13.111553 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.112008 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.112770 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.112952 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.113002 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:13.113195 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:13.113218 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:13.113250 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:13.113315 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 07:14:13.113330 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 07:14:13.113415 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:13.111182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:44147 (local address 127.10.130.193:58353)
0924 07:14:13.111312 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:13.111321 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:13.111370 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:13.111465 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:13.111465 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:13.111467 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 07:14:13.111513 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:13.111514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.112060 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.112060 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:13.112695 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:13.112698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.113046 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.113047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:13.113111 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:13.113124 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 07:14:13.113271 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 07:14:13.113272 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 07:14:13.113307 (+ 35us) client_negotiation.cc:240] Negotiation successful
0924 07:14:13.113343 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":66,"threads_started":1}
I20260924 07:14:13.115430 12920 raft_consensus.cc:1261] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Refusing update from remote peer 7f62c1d5c71742f1bee8647a1a532d1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:13.115433 12780 raft_consensus.cc:1261] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Refusing update from remote peer 7f62c1d5c71742f1bee8647a1a532d1d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:13.115657 13039 consensus_queue.cc:1035] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:13.115785 12994 consensus_queue.cc:1035] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:13.122200 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:13.120041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:45293 (local address 127.10.130.193:41533)
0924 07:14:13.120097 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:13.120098 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:13.120105 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:13.120122 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:13.120123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:13.120131 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:13.120178 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:13.120290 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.120863 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.121501 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.121744 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.121802 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:13.121977 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:13.122009 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:13.122053 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:13.122108 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 07:14:13.122135 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 07:14:13.122203 12832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:13.119860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:41533 (local address 127.10.130.194:45293)
0924 07:14:13.119897 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:13.119907 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:13.119946 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:13.120182 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:13.120183 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:13.120184 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 07:14:13.120225 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:13.120226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.120930 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.120931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:13.121441 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:13.121443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:13.121816 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:13.121817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:13.121871 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:13.121876 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 07:14:13.122058 (+ 182us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 07:14:13.122060 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:14:13.122096 (+ 36us) client_negotiation.cc:240] Negotiation successful
0924 07:14:13.122138 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
W20260924 07:14:13.122546 13014 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 07:14:14.639612 12482 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:14.639711 12482 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:14.640486 12482 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589: Sending DeleteTablet for 3 replicas of tablet 9ed93846d94f4e6d9fd3a6d9be3459a3
I20260924 07:14:14.640758 12900 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ed93846d94f4e6d9fd3a6d9be3459a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:14 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:50236
I20260924 07:14:14.640750 12760 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ed93846d94f4e6d9fd3a6d9be3459a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:14 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:40482
I20260924 07:14:14.640779 12620 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ed93846d94f4e6d9fd3a6d9be3459a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:14 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:60784
I20260924 07:14:14.640983 13071 tablet_replica.cc:330] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d: stopping tablet replica
I20260924 07:14:14.641043 13071 raft_consensus.cc:2229] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:14.641127 13071 raft_consensus.cc:2258] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:14.641356 13071 ts_tablet_manager.cc:1905] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:14.642798 13071 ts_tablet_manager.cc:1918] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 07:14:14.642849 13071 log.cc:1196] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/wal/wals/9ed93846d94f4e6d9fd3a6d9be3459a3
I20260924 07:14:14.643072 13071 ts_tablet_manager.cc:1939] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 7f62c1d5c71742f1bee8647a1a532d1d: Deleting consensus metadata
I20260924 07:14:14.643630 12468 catalog_manager.cc:4841] TS 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193:41533): tablet 9ed93846d94f4e6d9fd3a6d9be3459a3 (table test-table [id=c8e3c89964284f2790c12b8a52143c01]) successfully deleted
I20260924 07:14:14.644068 13069 tablet_replica.cc:330] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9: stopping tablet replica
I20260924 07:14:14.644073 13070 tablet_replica.cc:330] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521: stopping tablet replica
I20260924 07:14:14.644130 13069 raft_consensus.cc:2229] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:14.644130 13070 raft_consensus.cc:2229] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:14.644163 13069 raft_consensus.cc:2258] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:14.644181 13070 raft_consensus.cc:2258] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:14.644404 13070 ts_tablet_manager.cc:1905] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:14.644439 13069 ts_tablet_manager.cc:1905] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:14.645843 13069 ts_tablet_manager.cc:1918] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 07:14:14.645843 13070 ts_tablet_manager.cc:1918] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 07:14:14.645902 13070 log.cc:1196] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/9ed93846d94f4e6d9fd3a6d9be3459a3
I20260924 07:14:14.645902 13069 log.cc:1196] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/9ed93846d94f4e6d9fd3a6d9be3459a3
I20260924 07:14:14.646095 13069 ts_tablet_manager.cc:1939] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P 8753e90d9a07486ab5857e31c70d10e9: Deleting consensus metadata
I20260924 07:14:14.646095 13070 ts_tablet_manager.cc:1939] T 9ed93846d94f4e6d9fd3a6d9be3459a3 P f89cecb7c4734778b5a32aded7233521: Deleting consensus metadata
I20260924 07:14:14.646586 12468 catalog_manager.cc:4841] TS f89cecb7c4734778b5a32aded7233521 (127.10.130.194:44147): tablet 9ed93846d94f4e6d9fd3a6d9be3459a3 (table test-table [id=c8e3c89964284f2790c12b8a52143c01]) successfully deleted
I20260924 07:14:14.646683 12469 catalog_manager.cc:4841] TS 8753e90d9a07486ab5857e31c70d10e9 (127.10.130.195:38587): tablet 9ed93846d94f4e6d9fd3a6d9be3459a3 (table test-table [id=c8e3c89964284f2790c12b8a52143c01]) successfully deleted
Sep 24 07:14:14 dist-test-slave-7q7m krb5kdc[12438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234054, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 07:14:14 dist-test-slave-7q7m krb5kdc[12438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234054, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:14.663261 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:14.656624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47658 (local address 127.10.130.254:37223)
0924 07:14:14.656763 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:14.656765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:14.656774 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:14.656813 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:14.656813 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:14.656823 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:14.656877 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:14.657391 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:14.657878 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:14.658588 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:14.658766 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:14.658806 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:14.660942 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:14.660944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:14.660944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:14.660950 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:14.662261 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:14.662565 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:14.662566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:14.662567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:14.662618 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:14.662789 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:14.662789 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:14.662790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:14.662896 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:14.662981 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:14.663084 (+ 103us) server_negotiation.cc:299] Negotiation successful
0924 07:14:14.663167 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260924 07:14:14.664541 12482 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:14:14.664680 12482 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:14.665851 12900 tablet_service.cc:1467] Processing CreateTablet for tablet b5b51947fc3f42f6bb8952a989ab273f (DEFAULT_TABLE table=test-table [id=9047e8cfb94740749ae43777f8633372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:14.665860 12760 tablet_service.cc:1467] Processing CreateTablet for tablet b5b51947fc3f42f6bb8952a989ab273f (DEFAULT_TABLE table=test-table [id=9047e8cfb94740749ae43777f8633372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:14.665860 12620 tablet_service.cc:1467] Processing CreateTablet for tablet b5b51947fc3f42f6bb8952a989ab273f (DEFAULT_TABLE table=test-table [id=9047e8cfb94740749ae43777f8633372]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:14.665974 12760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5b51947fc3f42f6bb8952a989ab273f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:14.665974 12620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5b51947fc3f42f6bb8952a989ab273f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:14.665977 12900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5b51947fc3f42f6bb8952a989ab273f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:14.666991 13080 tablet_bootstrap.cc:492] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9: Bootstrap starting.
I20260924 07:14:14.667346 13080 tablet_bootstrap.cc:654] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:14.667338 13081 tablet_bootstrap.cc:492] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521: Bootstrap starting.
I20260924 07:14:14.667394 13082 tablet_bootstrap.cc:492] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d: Bootstrap starting.
I20260924 07:14:14.667699 13082 tablet_bootstrap.cc:654] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:14.667783 13081 tablet_bootstrap.cc:654] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:14.668012 13080 tablet_bootstrap.cc:492] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9: No bootstrap required, opened a new log
I20260924 07:14:14.668062 13080 ts_tablet_manager.cc:1397] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:14.668172 13080 raft_consensus.cc:348] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.668233 13080 raft_consensus.cc:374] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:14.668241 13082 tablet_bootstrap.cc:492] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d: No bootstrap required, opened a new log
I20260924 07:14:14.668251 13080 raft_consensus.cc:729] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8753e90d9a07486ab5857e31c70d10e9, State: Initialized, Role: FOLLOWER
I20260924 07:14:14.668275 13082 ts_tablet_manager.cc:1397] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:14.668313 13080 consensus_queue.cc:260] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.668397 13082 raft_consensus.cc:348] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.668465 13082 raft_consensus.cc:374] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:14.668483 13082 raft_consensus.cc:729] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62c1d5c71742f1bee8647a1a532d1d, State: Initialized, Role: FOLLOWER
I20260924 07:14:14.668524 13082 consensus_queue.cc:260] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.668469 13080 ts_tablet_manager.cc:1428] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:14.668633 13082 ts_tablet_manager.cc:1428] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:14.668864 13081 tablet_bootstrap.cc:492] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521: No bootstrap required, opened a new log
I20260924 07:14:14.668910 13081 ts_tablet_manager.cc:1397] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:14:14.669009 13081 raft_consensus.cc:348] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.669066 13081 raft_consensus.cc:374] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:14.669083 13081 raft_consensus.cc:729] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89cecb7c4734778b5a32aded7233521, State: Initialized, Role: FOLLOWER
I20260924 07:14:14.669126 13081 consensus_queue.cc:260] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.669205 13081 ts_tablet_manager.cc:1428] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:14.821602 13056 raft_consensus.cc:482] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:14.821699 13056 raft_consensus.cc:504] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.821870 13056 leader_election.cc:288] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193:41533), 8753e90d9a07486ab5857e31c70d10e9 (127.10.130.195:38587)
I20260924 07:14:14.822068 12920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b51947fc3f42f6bb8952a989ab273f" candidate_uuid: "f89cecb7c4734778b5a32aded7233521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8753e90d9a07486ab5857e31c70d10e9" is_pre_election: true
I20260924 07:14:14.822073 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b51947fc3f42f6bb8952a989ab273f" candidate_uuid: "f89cecb7c4734778b5a32aded7233521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" is_pre_election: true
I20260924 07:14:14.822151 12920 raft_consensus.cc:2454] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f89cecb7c4734778b5a32aded7233521 in term 0.
I20260924 07:14:14.822161 12640 raft_consensus.cc:2454] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f89cecb7c4734778b5a32aded7233521 in term 0.
I20260924 07:14:14.822304 12713 leader_election.cc:302] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8753e90d9a07486ab5857e31c70d10e9, f89cecb7c4734778b5a32aded7233521; no voters:
I20260924 07:14:14.822404 13056 raft_consensus.cc:2781] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:14.822451 13056 raft_consensus.cc:482] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:14.822463 13056 raft_consensus.cc:3037] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:14.823104 13056 raft_consensus.cc:504] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.823220 13056 leader_election.cc:288] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193:41533), 8753e90d9a07486ab5857e31c70d10e9 (127.10.130.195:38587)
I20260924 07:14:14.823360 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b51947fc3f42f6bb8952a989ab273f" candidate_uuid: "f89cecb7c4734778b5a32aded7233521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62c1d5c71742f1bee8647a1a532d1d"
I20260924 07:14:14.823367 12920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5b51947fc3f42f6bb8952a989ab273f" candidate_uuid: "f89cecb7c4734778b5a32aded7233521" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8753e90d9a07486ab5857e31c70d10e9"
I20260924 07:14:14.823437 12640 raft_consensus.cc:3037] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:14.823437 12920 raft_consensus.cc:3037] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:14.824234 12920 raft_consensus.cc:2454] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f89cecb7c4734778b5a32aded7233521 in term 1.
I20260924 07:14:14.824234 12640 raft_consensus.cc:2454] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f89cecb7c4734778b5a32aded7233521 in term 1.
I20260924 07:14:14.824421 12712 leader_election.cc:302] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62c1d5c71742f1bee8647a1a532d1d, f89cecb7c4734778b5a32aded7233521; no voters:
I20260924 07:14:14.824522 13056 raft_consensus.cc:2781] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:14.824569 13056 raft_consensus.cc:686] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [term 1 LEADER]: Becoming Leader. State: Replica: f89cecb7c4734778b5a32aded7233521, State: Running, Role: LEADER
I20260924 07:14:14.824618 13056 consensus_queue.cc:237] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.825208 12481 catalog_manager.cc:5495] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 reported cstate change: term changed from 0 to 1, leader changed from <none> to f89cecb7c4734778b5a32aded7233521 (127.10.130.194). New cstate: current_term: 1 leader_uuid: "f89cecb7c4734778b5a32aded7233521" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } health_report { overall_health: UNKNOWN } } }
I20260924 07:14:14.855661 13089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:14.853440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40492 (local address 127.10.130.194:44147)
0924 07:14:14.853587 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:14.853589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:14.853596 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:14.853614 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:14.853615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:14.853625 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:14.853696 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:14.853808 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:14.854323 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:14.855039 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:14.855242 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:14.855319 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:14.855397 (+ 78us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:14.855425 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:14.855499 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:14.855553 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 07:14:14.855576 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260924 07:14:14.859740 13089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:14.857885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:33835 (local address 127.10.130.194:44147)
0924 07:14:14.857921 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:14.857922 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:14.857933 (+ 11us) server_negotiation.cc:407] Connection header received
0924 07:14:14.857992 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:14.857993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:14.858002 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:14.858038 (+ 36us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:14.858127 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:14.858555 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:14.859226 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:14.859413 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:14.859468 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:14.859558 (+ 90us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:14.859579 (+ 21us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:14.859612 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:14.859653 (+ 41us) server_negotiation.cc:299] Negotiation successful
0924 07:14:14.859669 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:14:14.859751 13092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:14.857760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:44147 (local address 127.10.130.194:33835)
0924 07:14:14.857892 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:14.857901 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:14.857940 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:14.858050 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:14.858051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:14.858052 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 07:14:14.858086 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:14.858087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:14.858604 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:14.858605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:14.859135 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:14.859136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:14.859474 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:14.859475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:14.859509 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:14.859512 (+ 3us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 07:14:14.859621 (+ 109us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 07:14:14.859622 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 07:14:14.859652 (+ 30us) client_negotiation.cc:240] Negotiation successful
0924 07:14:14.859682 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260924 07:14:14.861325 12920 raft_consensus.cc:1261] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Refusing update from remote peer f89cecb7c4734778b5a32aded7233521: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:14.861346 12640 raft_consensus.cc:1261] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Refusing update from remote peer f89cecb7c4734778b5a32aded7233521: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:14.861500 13056 consensus_queue.cc:1035] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:14.861593 13048 consensus_queue.cc:1035] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:14.871259 12483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:14.871363 12483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:14.871939 12483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589: Sending DeleteTablet for 3 replicas of tablet b5b51947fc3f42f6bb8952a989ab273f
I20260924 07:14:14.872192 12900 tablet_service.cc:1514] Processing DeleteTablet for tablet b5b51947fc3f42f6bb8952a989ab273f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:14 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:50236
I20260924 07:14:14.872212 12760 tablet_service.cc:1514] Processing DeleteTablet for tablet b5b51947fc3f42f6bb8952a989ab273f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:14 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:40482
I20260924 07:14:14.872292 12620 tablet_service.cc:1514] Processing DeleteTablet for tablet b5b51947fc3f42f6bb8952a989ab273f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:14 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:60784
I20260924 07:14:14.872380 13069 tablet_replica.cc:330] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9: stopping tablet replica
I20260924 07:14:14.872383 13071 tablet_replica.cc:330] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d: stopping tablet replica
I20260924 07:14:14.872435 13071 raft_consensus.cc:2229] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:14.872435 13069 raft_consensus.cc:2229] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:14.872478 13069 raft_consensus.cc:2258] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:14.872474 13071 raft_consensus.cc:2258] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:14.872542 13070 tablet_replica.cc:330] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521: stopping tablet replica
I20260924 07:14:14.872602 13070 raft_consensus.cc:2229] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:14.872684 13070 raft_consensus.cc:2258] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:14.872816 13071 ts_tablet_manager.cc:1905] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:14.873095 13069 ts_tablet_manager.cc:1905] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:14.873209 13070 ts_tablet_manager.cc:1905] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:14.874492 13069 ts_tablet_manager.cc:1918] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:14.874505 13071 ts_tablet_manager.cc:1918] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:14.874545 13071 log.cc:1196] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-0/wal/wals/b5b51947fc3f42f6bb8952a989ab273f
I20260924 07:14:14.874545 13069 log.cc:1196] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/b5b51947fc3f42f6bb8952a989ab273f
I20260924 07:14:14.874620 13070 ts_tablet_manager.cc:1918] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:14.874668 13070 log.cc:1196] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/b5b51947fc3f42f6bb8952a989ab273f
I20260924 07:14:14.874745 13069 ts_tablet_manager.cc:1939] T b5b51947fc3f42f6bb8952a989ab273f P 8753e90d9a07486ab5857e31c70d10e9: Deleting consensus metadata
I20260924 07:14:14.874745 13071 ts_tablet_manager.cc:1939] T b5b51947fc3f42f6bb8952a989ab273f P 7f62c1d5c71742f1bee8647a1a532d1d: Deleting consensus metadata
I20260924 07:14:14.874861 13070 ts_tablet_manager.cc:1939] T b5b51947fc3f42f6bb8952a989ab273f P f89cecb7c4734778b5a32aded7233521: Deleting consensus metadata
I20260924 07:14:14.875062 12483 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 07:14:14.875121 12469 catalog_manager.cc:4841] TS 8753e90d9a07486ab5857e31c70d10e9 (127.10.130.195:38587): tablet b5b51947fc3f42f6bb8952a989ab273f (table test-table [id=9047e8cfb94740749ae43777f8633372]) successfully deleted
W20260924 07:14:14.875204 12483 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:14.875105 12468 catalog_manager.cc:4841] TS 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193:41533): tablet b5b51947fc3f42f6bb8952a989ab273f (table test-table [id=9047e8cfb94740749ae43777f8633372]) successfully deleted
I20260924 07:14:14.875651 12468 catalog_manager.cc:4841] TS f89cecb7c4734778b5a32aded7233521 (127.10.130.194:44147): tablet b5b51947fc3f42f6bb8952a989ab273f (table test-table [id=9047e8cfb94740749ae43777f8633372]) successfully deleted
I20260924 07:14:14.876312 12620 tablet_service.cc:1467] Processing CreateTablet for tablet 56c64a63ca74444fa3229c208c6fc43a (DEFAULT_TABLE table=test-table [id=16b20336d38e4d63a0cb856235b40286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:14.876315 12900 tablet_service.cc:1467] Processing CreateTablet for tablet 56c64a63ca74444fa3229c208c6fc43a (DEFAULT_TABLE table=test-table [id=16b20336d38e4d63a0cb856235b40286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:14.876411 12900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56c64a63ca74444fa3229c208c6fc43a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:14.876413 12620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56c64a63ca74444fa3229c208c6fc43a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:14.877173 13082 tablet_bootstrap.cc:492] T 56c64a63ca74444fa3229c208c6fc43a P 7f62c1d5c71742f1bee8647a1a532d1d: Bootstrap starting.
I20260924 07:14:14.877447 13082 tablet_bootstrap.cc:654] T 56c64a63ca74444fa3229c208c6fc43a P 7f62c1d5c71742f1bee8647a1a532d1d: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:14.877507 13080 tablet_bootstrap.cc:492] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9: Bootstrap starting.
I20260924 07:14:14.877780 13080 tablet_bootstrap.cc:654] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:14.877982 13082 tablet_bootstrap.cc:492] T 56c64a63ca74444fa3229c208c6fc43a P 7f62c1d5c71742f1bee8647a1a532d1d: No bootstrap required, opened a new log
I20260924 07:14:14.878027 13082 ts_tablet_manager.cc:1397] T 56c64a63ca74444fa3229c208c6fc43a P 7f62c1d5c71742f1bee8647a1a532d1d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:14.878136 13082 raft_consensus.cc:348] T 56c64a63ca74444fa3229c208c6fc43a P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.878197 13082 raft_consensus.cc:374] T 56c64a63ca74444fa3229c208c6fc43a P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:14.878181 12760 tablet_service.cc:1467] Processing CreateTablet for tablet 56c64a63ca74444fa3229c208c6fc43a (DEFAULT_TABLE table=test-table [id=16b20336d38e4d63a0cb856235b40286]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:14.878216 13082 raft_consensus.cc:729] T 56c64a63ca74444fa3229c208c6fc43a P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f62c1d5c71742f1bee8647a1a532d1d, State: Initialized, Role: FOLLOWER
I20260924 07:14:14.878269 12760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56c64a63ca74444fa3229c208c6fc43a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:14.878259 13082 consensus_queue.cc:260] T 56c64a63ca74444fa3229c208c6fc43a P 7f62c1d5c71742f1bee8647a1a532d1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.878345 13082 ts_tablet_manager.cc:1428] T 56c64a63ca74444fa3229c208c6fc43a P 7f62c1d5c71742f1bee8647a1a532d1d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:14.879377 13081 tablet_bootstrap.cc:492] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521: Bootstrap starting.
I20260924 07:14:14.879684 13081 tablet_bootstrap.cc:654] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:14.879926 13080 tablet_bootstrap.cc:492] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9: No bootstrap required, opened a new log
I20260924 07:14:14.879997 13080 ts_tablet_manager.cc:1397] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:14.880121 13080 raft_consensus.cc:348] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.880162 13081 tablet_bootstrap.cc:492] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521: No bootstrap required, opened a new log
I20260924 07:14:14.880189 13080 raft_consensus.cc:374] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:14.880200 13081 ts_tablet_manager.cc:1397] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 07:14:14.880209 13080 raft_consensus.cc:729] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8753e90d9a07486ab5857e31c70d10e9, State: Initialized, Role: FOLLOWER
I20260924 07:14:14.880249 13080 consensus_queue.cc:260] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.880287 13081 raft_consensus.cc:348] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.880329 13080 ts_tablet_manager.cc:1428] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:14.880350 13081 raft_consensus.cc:374] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:14.880374 13081 raft_consensus.cc:729] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89cecb7c4734778b5a32aded7233521, State: Initialized, Role: FOLLOWER
I20260924 07:14:14.880416 13081 consensus_queue.cc:260] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.880496 13081 ts_tablet_manager.cc:1428] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:14.908203 13086 raft_consensus.cc:482] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:14.908289 13086 raft_consensus.cc:504] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.908438 13086 leader_election.cc:288] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193:41533), f89cecb7c4734778b5a32aded7233521 (127.10.130.194:44147)
I20260924 07:14:14.908619 12780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56c64a63ca74444fa3229c208c6fc43a" candidate_uuid: "8753e90d9a07486ab5857e31c70d10e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f89cecb7c4734778b5a32aded7233521" is_pre_election: true
I20260924 07:14:14.908619 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56c64a63ca74444fa3229c208c6fc43a" candidate_uuid: "8753e90d9a07486ab5857e31c70d10e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" is_pre_election: true
I20260924 07:14:14.908704 12780 raft_consensus.cc:2454] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8753e90d9a07486ab5857e31c70d10e9 in term 0.
I20260924 07:14:14.908725 12640 raft_consensus.cc:2454] T 56c64a63ca74444fa3229c208c6fc43a P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8753e90d9a07486ab5857e31c70d10e9 in term 0.
I20260924 07:14:14.908859 12852 leader_election.cc:302] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f62c1d5c71742f1bee8647a1a532d1d, 8753e90d9a07486ab5857e31c70d10e9; no voters:
I20260924 07:14:14.908947 13086 raft_consensus.cc:2781] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:14.908994 13086 raft_consensus.cc:482] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:14.909013 13086 raft_consensus.cc:3037] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:14.909657 13086 raft_consensus.cc:504] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.909759 13086 leader_election.cc:288] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [CANDIDATE]: Term 1 election: Requested vote from peers 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193:41533), f89cecb7c4734778b5a32aded7233521 (127.10.130.194:44147)
I20260924 07:14:14.909909 12640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56c64a63ca74444fa3229c208c6fc43a" candidate_uuid: "8753e90d9a07486ab5857e31c70d10e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f62c1d5c71742f1bee8647a1a532d1d"
I20260924 07:14:14.909984 12640 raft_consensus.cc:3037] T 56c64a63ca74444fa3229c208c6fc43a P 7f62c1d5c71742f1bee8647a1a532d1d [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:14.909937 12780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56c64a63ca74444fa3229c208c6fc43a" candidate_uuid: "8753e90d9a07486ab5857e31c70d10e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f89cecb7c4734778b5a32aded7233521"
I20260924 07:14:14.910022 12780 raft_consensus.cc:3037] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:14.910539 12780 raft_consensus.cc:2454] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8753e90d9a07486ab5857e31c70d10e9 in term 1.
I20260924 07:14:14.910701 12852 leader_election.cc:302] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8753e90d9a07486ab5857e31c70d10e9, f89cecb7c4734778b5a32aded7233521; no voters:
I20260924 07:14:14.910722 12640 raft_consensus.cc:2454] T 56c64a63ca74444fa3229c208c6fc43a P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8753e90d9a07486ab5857e31c70d10e9 in term 1.
I20260924 07:14:14.910769 13086 raft_consensus.cc:2781] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:14.910876 13086 raft_consensus.cc:686] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [term 1 LEADER]: Becoming Leader. State: Replica: 8753e90d9a07486ab5857e31c70d10e9, State: Running, Role: LEADER
I20260924 07:14:14.910928 13086 consensus_queue.cc:237] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } }
I20260924 07:14:14.911473 12483 catalog_manager.cc:5495] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8753e90d9a07486ab5857e31c70d10e9 (127.10.130.195). New cstate: current_term: 1 leader_uuid: "8753e90d9a07486ab5857e31c70d10e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8753e90d9a07486ab5857e31c70d10e9" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38587 } health_report { overall_health: HEALTHY } } }
I20260924 07:14:14.926946 13101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:14.924370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50250 (local address 127.10.130.195:38587)
0924 07:14:14.924501 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:14.924503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:14.924510 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:14.924532 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:14.924533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:14.924543 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:14.924597 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:14.924727 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:14.925354 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:14.926291 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:14.926470 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:14.926510 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:14.926697 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:14.926734 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:14.926792 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:14.926865 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 07:14:14.926886 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260924 07:14:14.929243 12780 raft_consensus.cc:1261] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Refusing update from remote peer 8753e90d9a07486ab5857e31c70d10e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:14.929250 12640 raft_consensus.cc:1261] T 56c64a63ca74444fa3229c208c6fc43a P 7f62c1d5c71742f1bee8647a1a532d1d [term 1 FOLLOWER]: Refusing update from remote peer 8753e90d9a07486ab5857e31c70d10e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:14.929476 13099 consensus_queue.cc:1035] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f89cecb7c4734778b5a32aded7233521" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:14.929602 13086 consensus_queue.cc:1035] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f62c1d5c71742f1bee8647a1a532d1d" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 07:14:14.934182 13052 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 07:14:16.451596 12483 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:16.451715 12483 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:16.452524 12483 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d34275279e8e4780a03e7897ea52d589: Sending DeleteTablet for 3 replicas of tablet 56c64a63ca74444fa3229c208c6fc43a
I20260924 07:14:16.452797 12760 tablet_service.cc:1514] Processing DeleteTablet for tablet 56c64a63ca74444fa3229c208c6fc43a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:16 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:40482
I20260924 07:14:16.452802 12900 tablet_service.cc:1514] Processing DeleteTablet for tablet 56c64a63ca74444fa3229c208c6fc43a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:16 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:50236
I20260924 07:14:16.452963 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 12555
I20260924 07:14:16.452972 12620 tablet_service.cc:1514] Processing DeleteTablet for tablet 56c64a63ca74444fa3229c208c6fc43a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:16 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:60784
I20260924 07:14:16.453892 13129 tablet_replica.cc:330] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521: stopping tablet replica
I20260924 07:14:16.453948 13129 raft_consensus.cc:2229] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:16.453979 13129 raft_consensus.cc:2258] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:16.454205 13129 ts_tablet_manager.cc:1905] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:16.455928 13129 ts_tablet_manager.cc:1918] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 07:14:16.456012 13129 log.cc:1196] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/56c64a63ca74444fa3229c208c6fc43a
I20260924 07:14:16.456216 13129 ts_tablet_manager.cc:1939] T 56c64a63ca74444fa3229c208c6fc43a P f89cecb7c4734778b5a32aded7233521: Deleting consensus metadata
I20260924 07:14:16.456688 12468 catalog_manager.cc:4841] TS f89cecb7c4734778b5a32aded7233521 (127.10.130.194:44147): tablet 56c64a63ca74444fa3229c208c6fc43a (table test-table [id=16b20336d38e4d63a0cb856235b40286]) successfully deleted
I20260924 07:14:16.459125 13128 tablet_replica.cc:330] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9: stopping tablet replica
I20260924 07:14:16.459188 13128 raft_consensus.cc:2229] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:16.459250 13128 raft_consensus.cc:2258] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 07:14:16.459334 12468 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:41533 (error 108)
I20260924 07:14:16.459456 13128 ts_tablet_manager.cc:1905] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:16.459499 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 12695
I20260924 07:14:16.461269 13128 ts_tablet_manager.cc:1918] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 07:14:16.461328 13128 log.cc:1196] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/56c64a63ca74444fa3229c208c6fc43a
I20260924 07:14:16.461345 13131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:16.461114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:41533 (local address 127.0.0.1:60794)
0924 07:14:16.461248 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:16.461305 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:41533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
W20260924 07:14:16.461447 12468 catalog_manager.cc:4584] TS 7f62c1d5c71742f1bee8647a1a532d1d (127.10.130.193:41533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 56c64a63ca74444fa3229c208c6fc43a: Network error: Client connection negotiation failed: client connection to 127.10.130.193:41533: connect: Connection refused (error 111)
I20260924 07:14:16.461524 13128 ts_tablet_manager.cc:1939] T 56c64a63ca74444fa3229c208c6fc43a P 8753e90d9a07486ab5857e31c70d10e9: Deleting consensus metadata
I20260924 07:14:16.461845 12469 catalog_manager.cc:4841] TS 8753e90d9a07486ab5857e31c70d10e9 (127.10.130.195:38587): tablet 56c64a63ca74444fa3229c208c6fc43a (table test-table [id=16b20336d38e4d63a0cb856235b40286]) successfully deleted
I20260924 07:14:16.465001 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 12835
I20260924 07:14:16.469363 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 12451
2026-09-24T07:14:16Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5803 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:16 dist-test-slave-7q7m krb5kdc[13134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:16 dist-test-slave-7q7m krb5kdc[13134](info): set up 2 sockets
Sep 24 07:14:16 dist-test-slave-7q7m krb5kdc[13134](info): commencing operation
krb5kdc: starting...
W20260924 07:14:17.411408 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:17 dist-test-slave-7q7m krb5kdc[13134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:14:17.514505 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:42123
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:42123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:17.591831 13147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:17.592027 13147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:17.592048 13147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:17.593150 13147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:17.593189 13147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:17.593202 13147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:17.593215 13147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:17.593237 13147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:17.594619 13147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:42123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:42123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:17.594821 13147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:17.595026 13147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:17.599520 13155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:17.599576 13152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:17.599582 13153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:17.599807 13147 server_base.cc:1034] running on GCE node
I20260924 07:14:17.600055 13147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:17.600334 13147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:17.601471 13147 hybrid_clock.cc:648] HybridClock initialized: now 1790234057601448 us; error 34 us; skew 500 ppm
Sep 24 07:14:17 dist-test-slave-7q7m krb5kdc[13134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234057, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:17.603616 13147 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:14:17.603955 13147 webserver.cc:460] Webserver started at http://127.10.130.254:36959/ using document root <none> and password file <none>
I20260924 07:14:17.604159 13147 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:17.604202 13147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:17.604287 13147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:17.605175 13147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "0777ed25f71046f2bd6acbd7112c9a9c"
format_stamp: "Formatted at 2026-09-24 07:14:17 on dist-test-slave-7q7m"
server_key: "865664638def828dc858a28246ac3db4"
server_key_iv: "3d16f7fc4700c02ba9b5c9b1891b8eb7"
server_key_version: "encryptionkey@0"
I20260924 07:14:17.605440 13147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "0777ed25f71046f2bd6acbd7112c9a9c"
format_stamp: "Formatted at 2026-09-24 07:14:17 on dist-test-slave-7q7m"
server_key: "865664638def828dc858a28246ac3db4"
server_key_iv: "3d16f7fc4700c02ba9b5c9b1891b8eb7"
server_key_version: "encryptionkey@0"
I20260924 07:14:17.606510 13147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:14:17.607065 13162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:17.607214 13147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:17.607306 13147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "0777ed25f71046f2bd6acbd7112c9a9c"
format_stamp: "Formatted at 2026-09-24 07:14:17 on dist-test-slave-7q7m"
server_key: "865664638def828dc858a28246ac3db4"
server_key_iv: "3d16f7fc4700c02ba9b5c9b1891b8eb7"
server_key_version: "encryptionkey@0"
I20260924 07:14:17.607357 13147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:17.627621 13147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:17.629714 13147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:17.629818 13147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:17.633282 13147 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:42123
I20260924 07:14:17.633316 13214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:42123 every 8 connection(s)
I20260924 07:14:17.633642 13147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:17.634444 13215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:17.636514 13215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c: Bootstrap starting.
I20260924 07:14:17.637017 13215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:17.637231 13215 log.cc:824] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:17.637804 13215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c: No bootstrap required, opened a new log
I20260924 07:14:17.638484 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 13147
I20260924 07:14:17.638561 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:17.638698 10763 external_mini_cluster.cc:1428] Setting key ac7c4e49a7c5a8a7e27288a86c86179e
I20260924 07:14:17.639642 13215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0777ed25f71046f2bd6acbd7112c9a9c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42123 } }
I20260924 07:14:17.639802 13215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:17.639837 13215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0777ed25f71046f2bd6acbd7112c9a9c, State: Initialized, Role: FOLLOWER
I20260924 07:14:17.640005 13215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0777ed25f71046f2bd6acbd7112c9a9c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42123 } }
I20260924 07:14:17.640085 13215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:17.640107 13215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:17.640154 13215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:17.641091 13215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0777ed25f71046f2bd6acbd7112c9a9c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42123 } }
I20260924 07:14:17.641191 13215 leader_election.cc:302] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0777ed25f71046f2bd6acbd7112c9a9c; no voters:
I20260924 07:14:17.641328 13215 leader_election.cc:288] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:17.641371 13220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:17.641496 13220 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [term 1 LEADER]: Becoming Leader. State: Replica: 0777ed25f71046f2bd6acbd7112c9a9c, State: Running, Role: LEADER
I20260924 07:14:17.641561 13215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:17.641623 13220 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0777ed25f71046f2bd6acbd7112c9a9c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42123 } }
I20260924 07:14:17.641955 13220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0777ed25f71046f2bd6acbd7112c9a9c. Latest consensus state: current_term: 1 leader_uuid: "0777ed25f71046f2bd6acbd7112c9a9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0777ed25f71046f2bd6acbd7112c9a9c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42123 } } }
I20260924 07:14:17.642201 13220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:17.642596 13229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:17.643054 13229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:17.642112 13221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0777ed25f71046f2bd6acbd7112c9a9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0777ed25f71046f2bd6acbd7112c9a9c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42123 } } }
I20260924 07:14:17.643703 13221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:17.644701 13229 catalog_manager.cc:1347] Generated new cluster ID: 6097c1f68d4e41ddaa56f3579c24cb27
I20260924 07:14:17.644747 13229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:14:17 dist-test-slave-7q7m krb5kdc[13134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:17.649817 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:17.639156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49234 (local address 127.10.130.254:42123)
0924 07:14:17.639348 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:17.639358 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:17.639392 (+ 34us) server_negotiation.cc:407] Connection header received
0924 07:14:17.639623 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:17.639631 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:17.639693 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:17.639833 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:17.640514 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:17.641871 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:17.642955 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:17.643528 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:17.643636 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:17.646381 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:17.646385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:17.646386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:17.646396 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:17.648297 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:17.648653 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:17.648657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:17.648657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:17.648790 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:17.648968 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:17.648969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:17.648969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:17.649269 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:17.649424 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:17.649489 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 07:14:17.649622 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260924 07:14:17.665388 13229 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:17.665896 13229 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:17.671424 13229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0777ed25f71046f2bd6acbd7112c9a9c: Generated new TSK 0
I20260924 07:14:17.671602 13229 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:14:17.696664 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42123
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:17.773869 13241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:17.774058 13241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:17.774078 13241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:17.775187 13241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:17.775218 13241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:17.775240 13241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:17.776615 13241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:17.776803 13241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:17.776980 13241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:17.781576 13249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:17.781683 13241 server_base.cc:1034] running on GCE node
W20260924 07:14:17.781550 13246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:17.781558 13247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:17.781991 13241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:17.782217 13241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:17.783358 13241 hybrid_clock.cc:648] HybridClock initialized: now 1790234057783340 us; error 33 us; skew 500 ppm
Sep 24 07:14:17 dist-test-slave-7q7m krb5kdc[13134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234057, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:17.785596 13241 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:14:17.785993 13241 webserver.cc:460] Webserver started at http://127.10.130.193:43651/ using document root <none> and password file <none>
I20260924 07:14:17.786178 13241 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:17.786217 13241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:17.786304 13241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:17.787269 13241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "a6ba6a9e061240a48d8902216ddbf2d5"
format_stamp: "Formatted at 2026-09-24 07:14:17 on dist-test-slave-7q7m"
server_key: "e3cf9978c58320e33aea7304d1222e70"
server_key_iv: "7a7fc636bc038e29fac90a78473e4a4b"
server_key_version: "encryptionkey@0"
I20260924 07:14:17.787544 13241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "a6ba6a9e061240a48d8902216ddbf2d5"
format_stamp: "Formatted at 2026-09-24 07:14:17 on dist-test-slave-7q7m"
server_key: "e3cf9978c58320e33aea7304d1222e70"
server_key_iv: "7a7fc636bc038e29fac90a78473e4a4b"
server_key_version: "encryptionkey@0"
I20260924 07:14:17.788740 13241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:14:17.789325 13256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:17.789463 13241 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:17.789558 13241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "a6ba6a9e061240a48d8902216ddbf2d5"
format_stamp: "Formatted at 2026-09-24 07:14:17 on dist-test-slave-7q7m"
server_key: "e3cf9978c58320e33aea7304d1222e70"
server_key_iv: "7a7fc636bc038e29fac90a78473e4a4b"
server_key_version: "encryptionkey@0"
I20260924 07:14:17.789624 13241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:17.800824 13241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:17.802625 13241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:17.802740 13241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:17.802942 13241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:17.803236 13241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:17.803270 13241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:17.803292 13241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:17.803315 13241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:17.808611 13241 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:36977
I20260924 07:14:17.808670 13369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:36977 every 8 connection(s)
I20260924 07:14:17.808933 13241 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:14:17.810810 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 13241
I20260924 07:14:17.810883 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:17.811028 10763 external_mini_cluster.cc:1428] Setting key c9e5b352efa90ac910c0592efb08045a
Sep 24 07:14:17 dist-test-slave-7q7m krb5kdc[13134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234057, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:17.818120 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:17.809632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:46147 (local address 127.10.130.254:42123)
0924 07:14:17.809673 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:17.809675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:17.809845 (+ 170us) server_negotiation.cc:407] Connection header received
0924 07:14:17.810298 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:17.810299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:17.810309 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:17.810356 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:17.811518 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:17.812223 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:17.813200 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:17.813405 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:17.813463 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:17.815531 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:17.815532 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:17.815533 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:17.815542 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:17.816873 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:17.817255 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:17.817257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:17.817257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:17.817310 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:17.817526 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:17.817527 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:17.817527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:17.817631 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:17.817683 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:17.817937 (+ 254us) server_negotiation.cc:299] Negotiation successful
0924 07:14:17.818030 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
I20260924 07:14:17.818254 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:17.809586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:42123 (local address 127.10.130.193:46147)
0924 07:14:17.809746 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:17.809780 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:17.810230 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:17.810410 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:17.810413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:17.811042 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:17.811260 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:17.811265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:17.812304 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:17.812305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:17.813110 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:17.813112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:17.813479 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:17.813480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:17.813606 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:17.814087 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:17.814095 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:17.815440 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:17.816967 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:17.816973 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:17.816976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:17.817184 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:17.817370 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:17.817371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:17.817372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:17.817467 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:17.817689 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:17.817694 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:17.817821 (+ 127us) client_negotiation.cc:769] Sending connection context
0924 07:14:17.817905 (+ 84us) client_negotiation.cc:240] Negotiation successful
0924 07:14:17.818051 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
I20260924 07:14:17.818681 13370 heartbeater.cc:347] Connected to a master server at 127.10.130.254:42123
I20260924 07:14:17.818769 13370 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:17.818934 13370 heartbeater.cc:511] Master 127.10.130.254:42123 requested a full tablet report, sending...
I20260924 07:14:17.819455 13179 ts_manager.cc:194] Registered new tserver with Master: a6ba6a9e061240a48d8902216ddbf2d5 (127.10.130.193:36977)
I20260924 07:14:17.820168 13179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:46147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:14:17.832186 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42123
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:17.909309 13375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:17.909505 13375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:17.909535 13375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:17.910641 13375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:17.910681 13375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:17.910714 13375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:17.912099 13375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:17.912308 13375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:17.912513 13375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:17.916996 13383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:17.917044 13381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:17.917084 13380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:17.917088 13375 server_base.cc:1034] running on GCE node
I20260924 07:14:17.917374 13375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:17.917584 13375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:17.918708 13375 hybrid_clock.cc:648] HybridClock initialized: now 1790234057918692 us; error 32 us; skew 500 ppm
Sep 24 07:14:17 dist-test-slave-7q7m krb5kdc[13134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234057, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:17.921006 13375 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:14:17.921382 13375 webserver.cc:460] Webserver started at http://127.10.130.194:34461/ using document root <none> and password file <none>
I20260924 07:14:17.921569 13375 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:17.921615 13375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:17.921737 13375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:17.922696 13375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "21dbd4ec76b0431cad4fb2c0367b4062"
format_stamp: "Formatted at 2026-09-24 07:14:17 on dist-test-slave-7q7m"
server_key: "37fad7663caf0397e907d52d21607cfd"
server_key_iv: "4b90a25465a461b05395aa40ffb49884"
server_key_version: "encryptionkey@0"
I20260924 07:14:17.923004 13375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "21dbd4ec76b0431cad4fb2c0367b4062"
format_stamp: "Formatted at 2026-09-24 07:14:17 on dist-test-slave-7q7m"
server_key: "37fad7663caf0397e907d52d21607cfd"
server_key_iv: "4b90a25465a461b05395aa40ffb49884"
server_key_version: "encryptionkey@0"
I20260924 07:14:17.924172 13375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:14:17.924782 13390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:17.924930 13375 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:17.925006 13375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "21dbd4ec76b0431cad4fb2c0367b4062"
format_stamp: "Formatted at 2026-09-24 07:14:17 on dist-test-slave-7q7m"
server_key: "37fad7663caf0397e907d52d21607cfd"
server_key_iv: "4b90a25465a461b05395aa40ffb49884"
server_key_version: "encryptionkey@0"
I20260924 07:14:17.925069 13375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:17.947283 13375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:17.949084 13375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:17.949221 13375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:17.949431 13375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:17.949714 13375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:17.949746 13375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:17.949776 13375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:17.949793 13375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:17.954983 13375 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:35771
I20260924 07:14:17.955055 13503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:35771 every 8 connection(s)
I20260924 07:14:17.955315 13375 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:14:17.955864 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 13375
I20260924 07:14:17.955953 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:17.956158 10763 external_mini_cluster.cc:1428] Setting key 1dd0fd4c168529bdc32dff070b4a56d7
Sep 24 07:14:17 dist-test-slave-7q7m krb5kdc[13134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234057, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
I20260924 07:14:17.964957 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:17.956008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:46787 (local address 127.10.130.254:42123)
0924 07:14:17.956063 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:17.956065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:17.956296 (+ 231us) server_negotiation.cc:407] Connection header received
0924 07:14:17.956846 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:17.956847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:17.956857 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:17.956908 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:17.958039 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:17.958532 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:17.959881 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:17.960098 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:17.960142 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:17.962359 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:17.962361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:17.962361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:17.962368 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:17.963554 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:17.964038 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:17.964039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:17.964039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:17.964084 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:17.964368 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:17.964369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:17.964370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:17.964500 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:17.964574 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:17.964827 (+ 253us) server_negotiation.cc:299] Negotiation successful
0924 07:14:17.964894 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 07:14:17.965077 13506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:17.955966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:42123 (local address 127.10.130.194:46787)
0924 07:14:17.956159 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:17.956218 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:17.956757 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:17.957000 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:17.957004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:17.957697 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:17.957959 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:17.957965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:17.958616 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:17.958617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:17.959806 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:17.959809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:17.960164 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:17.960165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:17.960305 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:17.960752 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:17.960762 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:17.962271 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:17.963636 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:17.963642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:17.963645 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:17.963929 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:17.964143 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:17.964143 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:17.964144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:17.964274 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:17.964578 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:17.964583 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:17.964711 (+ 128us) client_negotiation.cc:769] Sending connection context
0924 07:14:17.964795 (+ 84us) client_negotiation.cc:240] Negotiation successful
0924 07:14:17.964909 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260924 07:14:17.965498 13504 heartbeater.cc:347] Connected to a master server at 127.10.130.254:42123
I20260924 07:14:17.965608 13504 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:17.965775 13504 heartbeater.cc:511] Master 127.10.130.254:42123 requested a full tablet report, sending...
I20260924 07:14:17.966157 13179 ts_manager.cc:194] Registered new tserver with Master: 21dbd4ec76b0431cad4fb2c0367b4062 (127.10.130.194:35771)
I20260924 07:14:17.966485 13179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:46787
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:14:17.978856 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42123
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:40525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:18.055915 13509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:18.056129 13509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:18.056150 13509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:18.057284 13509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:18.057325 13509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:18.057353 13509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:18.058737 13509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:40525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:18.058943 13509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:18.059152 13509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 07:14:18.063967 13509 server_base.cc:1034] running on GCE node
W20260924 07:14:18.063943 13517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:18.063941 13515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:18.063933 13514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:18.064337 13509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:18.064579 13509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:18.065721 13509 hybrid_clock.cc:648] HybridClock initialized: now 1790234058065708 us; error 25 us; skew 500 ppm
Sep 24 07:14:18 dist-test-slave-7q7m krb5kdc[13134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234058, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:18.067993 13509 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:14:18.068385 13509 webserver.cc:460] Webserver started at http://127.10.130.195:39987/ using document root <none> and password file <none>
I20260924 07:14:18.068576 13509 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:18.068622 13509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:18.068750 13509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:18.069754 13509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "b5170d8bf06c4a06bb72bcd0c1bdfe3d"
format_stamp: "Formatted at 2026-09-24 07:14:18 on dist-test-slave-7q7m"
server_key: "0168b6c98970a7432045ddc39bb116bd"
server_key_iv: "0ba3e6472b398de8a6ee1577c2b59ce3"
server_key_version: "encryptionkey@0"
I20260924 07:14:18.070086 13509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "b5170d8bf06c4a06bb72bcd0c1bdfe3d"
format_stamp: "Formatted at 2026-09-24 07:14:18 on dist-test-slave-7q7m"
server_key: "0168b6c98970a7432045ddc39bb116bd"
server_key_iv: "0ba3e6472b398de8a6ee1577c2b59ce3"
server_key_version: "encryptionkey@0"
I20260924 07:14:18.071278 13509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260924 07:14:18.071906 13524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:18.072068 13509 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:18.072147 13509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "b5170d8bf06c4a06bb72bcd0c1bdfe3d"
format_stamp: "Formatted at 2026-09-24 07:14:18 on dist-test-slave-7q7m"
server_key: "0168b6c98970a7432045ddc39bb116bd"
server_key_iv: "0ba3e6472b398de8a6ee1577c2b59ce3"
server_key_version: "encryptionkey@0"
I20260924 07:14:18.072213 13509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:18.090634 13509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:18.092620 13509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:18.092762 13509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:18.092988 13509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:18.093343 13509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:18.093385 13509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:18.093415 13509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:18.093438 13509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:18.098722 13509 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:39943
I20260924 07:14:18.098774 13637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:39943 every 8 connection(s)
I20260924 07:14:18.099071 13509 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:18.102870 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 13509
I20260924 07:14:18.102973 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:18.103174 10763 external_mini_cluster.cc:1428] Setting key 2b429ce3a35a8d690a6ff7e9b19b3c97
Sep 24 07:14:18 dist-test-slave-7q7m krb5kdc[13134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234058, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:18.107925 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:18.099769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:50201 (local address 127.10.130.254:42123)
0924 07:14:18.099847 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:18.099849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:18.100028 (+ 179us) server_negotiation.cc:407] Connection header received
0924 07:14:18.100569 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:18.100570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:18.100583 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:18.100647 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:18.101587 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:18.102082 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:18.103248 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:18.103519 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:18.103578 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:18.105759 (+ 2181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:18.105761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:18.105761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:18.105767 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:18.106863 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:18.107217 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:18.107218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:18.107218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:18.107278 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:18.107473 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:18.107473 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:18.107473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:18.107558 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:18.107599 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:18.107808 (+ 209us) server_negotiation.cc:299] Negotiation successful
0924 07:14:18.107864 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 07:14:18.108129 13640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:18.099698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:42123 (local address 127.10.130.195:50201)
0924 07:14:18.099884 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:18.099925 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:18.100445 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:18.100727 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:18.100731 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:18.101290 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:18.101477 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:18.101482 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:18.102170 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:18.102172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:18.103158 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:18.103161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:18.103624 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:18.103626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:18.103804 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:18.104351 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:18.104359 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:18.105672 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:18.106925 (+ 1253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:18.106931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:18.106932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:18.107155 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:18.107329 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:18.107330 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:18.107330 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:18.107423 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:18.107602 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:18.107604 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:18.107709 (+ 105us) client_negotiation.cc:769] Sending connection context
0924 07:14:18.107794 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 07:14:18.107908 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
I20260924 07:14:18.108566 13638 heartbeater.cc:347] Connected to a master server at 127.10.130.254:42123
I20260924 07:14:18.108646 13638 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:18.108811 13638 heartbeater.cc:511] Master 127.10.130.254:42123 requested a full tablet report, sending...
I20260924 07:14:18.109148 13179 ts_manager.cc:194] Registered new tserver with Master: b5170d8bf06c4a06bb72bcd0c1bdfe3d (127.10.130.195:39943)
I20260924 07:14:18.109572 13179 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:50201
I20260924 07:14:18.114795 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 07:14:18.124616 13219 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:18.123727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52968 (local address 127.10.130.254:42123)
0924 07:14:18.123787 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:18.123789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:18.123897 (+ 108us) server_negotiation.cc:407] Connection header received
0924 07:14:18.123966 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:18.123967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:18.123992 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:18.124050 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:18.124547 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52968: BlockingRecv error: recv got EOF from 127.0.0.1:52968 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 07:14:18.125474 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 13241
I20260924 07:14:18.129580 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 13375
I20260924 07:14:18.133311 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 13509
I20260924 07:14:18.137061 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 13147
2026-09-24T07:14:18Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1665 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:18 dist-test-slave-7q7m krb5kdc[13650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:18 dist-test-slave-7q7m krb5kdc[13650](info): set up 2 sockets
Sep 24 07:14:18 dist-test-slave-7q7m krb5kdc[13650](info): commencing operation
krb5kdc: starting...
W20260924 07:14:19.087448 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:19 dist-test-slave-7q7m krb5kdc[13650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:14:19.190114 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:33547
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:33547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:19.266115 13663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:19.266307 13663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:19.266335 13663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:19.267472 13663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:19.267518 13663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:19.267537 13663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:19.267553 13663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:19.267568 13663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:19.269021 13663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:33547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:33547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:19.269250 13663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:19.269445 13663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 07:14:19.273963 13663 server_base.cc:1034] running on GCE node
W20260924 07:14:19.273904 13671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:19.273921 13669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:19.273931 13668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:19.274355 13663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:19.274605 13663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:19.275748 13663 hybrid_clock.cc:648] HybridClock initialized: now 1790234059275735 us; error 28 us; skew 500 ppm
Sep 24 07:14:19 dist-test-slave-7q7m krb5kdc[13650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234059, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:19.278024 13663 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:14:19.278403 13663 webserver.cc:460] Webserver started at http://127.10.130.254:41561/ using document root <none> and password file <none>
I20260924 07:14:19.278596 13663 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:19.278641 13663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:19.278760 13663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:19.279738 13663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "bb4526565ba8454394c76d20a62ea759"
format_stamp: "Formatted at 2026-09-24 07:14:19 on dist-test-slave-7q7m"
server_key: "c34f81eb10406b0567d062b940636c9b"
server_key_iv: "f084426914c7ded652d2ba0d75aa764b"
server_key_version: "encryptionkey@0"
I20260924 07:14:19.280095 13663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "bb4526565ba8454394c76d20a62ea759"
format_stamp: "Formatted at 2026-09-24 07:14:19 on dist-test-slave-7q7m"
server_key: "c34f81eb10406b0567d062b940636c9b"
server_key_iv: "f084426914c7ded652d2ba0d75aa764b"
server_key_version: "encryptionkey@0"
I20260924 07:14:19.281244 13663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:19.281858 13678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:19.282020 13663 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:19.282097 13663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "bb4526565ba8454394c76d20a62ea759"
format_stamp: "Formatted at 2026-09-24 07:14:19 on dist-test-slave-7q7m"
server_key: "c34f81eb10406b0567d062b940636c9b"
server_key_iv: "f084426914c7ded652d2ba0d75aa764b"
server_key_version: "encryptionkey@0"
I20260924 07:14:19.282160 13663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:19.297384 13663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:19.299232 13663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:19.299353 13663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:19.302829 13663 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:33547
I20260924 07:14:19.302861 13730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:33547 every 8 connection(s)
I20260924 07:14:19.303267 13663 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:19.303794 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 13663
I20260924 07:14:19.303860 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:19.303843 13731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:19.304036 10763 external_mini_cluster.cc:1428] Setting key e965abc13a6a412f4dfa48936a4946b1
I20260924 07:14:19.306857 13731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759: Bootstrap starting.
I20260924 07:14:19.307512 13731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:19.307754 13731 log.cc:824] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:19.308367 13731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759: No bootstrap required, opened a new log
Sep 24 07:14:19 dist-test-slave-7q7m krb5kdc[13650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:19.310570 13731 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4526565ba8454394c76d20a62ea759" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33547 } }
I20260924 07:14:19.310702 13731 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:19.310745 13731 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb4526565ba8454394c76d20a62ea759, State: Initialized, Role: FOLLOWER
I20260924 07:14:19.310860 13731 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4526565ba8454394c76d20a62ea759" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33547 } }
I20260924 07:14:19.310930 13731 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:19.310958 13731 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:19.311002 13731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:19.311580 13731 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4526565ba8454394c76d20a62ea759" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33547 } }
I20260924 07:14:19.311698 13731 leader_election.cc:302] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb4526565ba8454394c76d20a62ea759; no voters:
I20260924 07:14:19.311849 13731 leader_election.cc:288] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:19.311892 13736 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:19.312075 13736 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [term 1 LEADER]: Becoming Leader. State: Replica: bb4526565ba8454394c76d20a62ea759, State: Running, Role: LEADER
I20260924 07:14:19.312095 13731 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:19.312233 13736 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4526565ba8454394c76d20a62ea759" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33547 } }
I20260924 07:14:19.313405 13737 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb4526565ba8454394c76d20a62ea759. Latest consensus state: current_term: 1 leader_uuid: "bb4526565ba8454394c76d20a62ea759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4526565ba8454394c76d20a62ea759" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33547 } } }
I20260924 07:14:19.313534 13737 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:19.313421 13738 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb4526565ba8454394c76d20a62ea759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb4526565ba8454394c76d20a62ea759" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33547 } } }
I20260924 07:14:19.313849 13738 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759 [sys.catalog]: This master's current role is: LEADER
W20260924 07:14:19.313859 13751 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:14:19.314003 13751 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:14:19.314193 13734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:19.304461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48362 (local address 127.10.130.254:33547)
0924 07:14:19.304648 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:19.304657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:19.304688 (+ 31us) server_negotiation.cc:407] Connection header received
0924 07:14:19.305257 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:19.305267 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:19.305327 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:19.305471 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:19.305974 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:19.307302 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:19.308044 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:19.308509 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:19.308590 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:19.310825 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:19.310829 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:19.310830 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:19.310840 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:19.312551 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:19.312893 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:19.312897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:19.312897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:19.312997 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:19.313219 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:19.313220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:19.313220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:19.313507 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:19.313630 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:19.313862 (+ 232us) server_negotiation.cc:299] Negotiation successful
0924 07:14:19.313986 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260924 07:14:19.314280 13752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:19.314410 13752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:19.314954 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:14:19.315970 13752 catalog_manager.cc:1347] Generated new cluster ID: af25da241ed642019f2709ef2ee9a5fc
I20260924 07:14:19.316041 13752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:19.329737 13734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:19.326738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48364 (local address 127.10.130.254:33547)
0924 07:14:19.326774 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:19.326776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:19.326898 (+ 122us) server_negotiation.cc:407] Connection header received
0924 07:14:19.326958 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:19.326959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:19.326968 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:19.327009 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:19.327349 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:19.327803 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:19.328531 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:19.328739 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:19.328792 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:19.329350 (+ 558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:19.329352 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:19.329352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:19.329516 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:19.329529 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:19.329534 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:19.329579 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:19.329637 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 07:14:19.329658 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:14:19.355521 13752 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:19.356281 13752 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:19.362059 13752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb4526565ba8454394c76d20a62ea759: Generated new TSK 0
I20260924 07:14:19.362217 13752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:14:19.368536 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 13663
2026-09-24T07:14:19Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1230 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:19 dist-test-slave-7q7m krb5kdc[13764](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:19 dist-test-slave-7q7m krb5kdc[13764](info): set up 2 sockets
Sep 24 07:14:19 dist-test-slave-7q7m krb5kdc[13764](info): commencing operation
krb5kdc: starting...
W20260924 07:14:20.299667 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.914s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:20 dist-test-slave-7q7m krb5kdc[13764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:14:20.402565 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:42585
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:40627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:42585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:20.481101 13777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:20.481289 13777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:20.481320 13777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:20.482411 13777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:20.482448 13777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:20.482466 13777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:20.482482 13777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:20.482497 13777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:20.483912 13777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:40627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:42585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:42585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:20.484155 13777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:20.484356 13777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:20.488765 13783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:20.488745 13785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:20.488874 13777 server_base.cc:1034] running on GCE node
W20260924 07:14:20.488947 13782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:20.489272 13777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:20.489564 13777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:20.490707 13777 hybrid_clock.cc:648] HybridClock initialized: now 1790234060490697 us; error 31 us; skew 500 ppm
Sep 24 07:14:20 dist-test-slave-7q7m krb5kdc[13764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234060, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:20.492949 13777 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:14:20.493309 13777 webserver.cc:460] Webserver started at http://127.10.130.254:39531/ using document root <none> and password file <none>
I20260924 07:14:20.493497 13777 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:20.493541 13777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:20.493664 13777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:20.494585 13777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "b7287819fa5a43c0be01106c1946792d"
format_stamp: "Formatted at 2026-09-24 07:14:20 on dist-test-slave-7q7m"
server_key: "6350a1474ae5c6e5215455d0f512681c"
server_key_iv: "87b1e8b191cc7ad2afdd89358ec8ff7a"
server_key_version: "encryptionkey@0"
I20260924 07:14:20.494900 13777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "b7287819fa5a43c0be01106c1946792d"
format_stamp: "Formatted at 2026-09-24 07:14:20 on dist-test-slave-7q7m"
server_key: "6350a1474ae5c6e5215455d0f512681c"
server_key_iv: "87b1e8b191cc7ad2afdd89358ec8ff7a"
server_key_version: "encryptionkey@0"
I20260924 07:14:20.496062 13777 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:14:20.496665 13792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:20.496801 13777 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:20.496876 13777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "b7287819fa5a43c0be01106c1946792d"
format_stamp: "Formatted at 2026-09-24 07:14:20 on dist-test-slave-7q7m"
server_key: "6350a1474ae5c6e5215455d0f512681c"
server_key_iv: "87b1e8b191cc7ad2afdd89358ec8ff7a"
server_key_version: "encryptionkey@0"
I20260924 07:14:20.496933 13777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:20.508826 13777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:20.510433 13777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:20.510552 13777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:20.513914 13777 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:42585
I20260924 07:14:20.513947 13844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:42585 every 8 connection(s)
I20260924 07:14:20.514235 13777 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:20.515086 13845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:20.516086 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 13777
I20260924 07:14:20.516170 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:20.516323 10763 external_mini_cluster.cc:1428] Setting key 497a8b6d60cfeccf0b7e7ffadf384236
I20260924 07:14:20.517655 13845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d: Bootstrap starting.
I20260924 07:14:20.518234 13845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:20.518515 13845 log.cc:824] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:20.519219 13845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d: No bootstrap required, opened a new log
I20260924 07:14:20.521332 13845 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7287819fa5a43c0be01106c1946792d" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42585 } }
I20260924 07:14:20.521462 13845 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:20.521482 13845 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7287819fa5a43c0be01106c1946792d, State: Initialized, Role: FOLLOWER
I20260924 07:14:20.521588 13845 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7287819fa5a43c0be01106c1946792d" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42585 } }
I20260924 07:14:20.521667 13845 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:20.521688 13845 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:20.521733 13845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:20.522265 13845 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7287819fa5a43c0be01106c1946792d" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42585 } }
I20260924 07:14:20.522387 13845 leader_election.cc:302] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7287819fa5a43c0be01106c1946792d; no voters:
I20260924 07:14:20.522544 13845 leader_election.cc:288] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:20.522612 13850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [term 1 FOLLOWER]: Leader election won for term 1
Sep 24 07:14:20 dist-test-slave-7q7m krb5kdc[13764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:20.522784 13845 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:20.522846 13850 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [term 1 LEADER]: Becoming Leader. State: Replica: b7287819fa5a43c0be01106c1946792d, State: Running, Role: LEADER
I20260924 07:14:20.523054 13850 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7287819fa5a43c0be01106c1946792d" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42585 } }
I20260924 07:14:20.523362 13852 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7287819fa5a43c0be01106c1946792d. Latest consensus state: current_term: 1 leader_uuid: "b7287819fa5a43c0be01106c1946792d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7287819fa5a43c0be01106c1946792d" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42585 } } }
I20260924 07:14:20.523473 13852 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:20.523362 13851 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7287819fa5a43c0be01106c1946792d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7287819fa5a43c0be01106c1946792d" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42585 } } }
I20260924 07:14:20.523700 13851 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:20.523844 13859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:20.524344 13859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:20.525883 13859 catalog_manager.cc:1347] Generated new cluster ID: 710274cacc0d4be98e1babd0013607eb
I20260924 07:14:20.525996 13859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:20.526721 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:20.516751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38968 (local address 127.10.130.254:42585)
0924 07:14:20.516921 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:20.516930 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:20.516962 (+ 32us) server_negotiation.cc:407] Connection header received
0924 07:14:20.517426 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:20.517435 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:20.517495 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:20.517641 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:20.518176 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.519418 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.520150 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.520750 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.520874 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:20.523315 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:20.523319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:20.523319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:20.523330 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:20.525174 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:20.525549 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:20.525552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:20.525553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:20.525661 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:20.525877 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:20.525878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:20.525879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:20.526151 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:20.526284 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:20.526398 (+ 114us) server_negotiation.cc:299] Negotiation successful
0924 07:14:20.526525 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260924 07:14:20.538702 13859 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:20.539368 13859 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:20.545459 13859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7287819fa5a43c0be01106c1946792d: Generated new TSK 0
I20260924 07:14:20.545606 13859 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:14:20.571514 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42585
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:40627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:20.647166 13871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:20.647363 13871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:20.647385 13871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:20.648499 13871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:20.648533 13871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:20.648562 13871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:20.649902 13871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:40627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:20.650111 13871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:20.650305 13871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:20.654721 13876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:20.654886 13871 server_base.cc:1034] running on GCE node
W20260924 07:14:20.654726 13877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:20.654791 13879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:20.655177 13871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:20.655390 13871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:20.656512 13871 hybrid_clock.cc:648] HybridClock initialized: now 1790234060656497 us; error 31 us; skew 500 ppm
Sep 24 07:14:20 dist-test-slave-7q7m krb5kdc[13764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234060, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:20.658660 13871 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:14:20.659008 13871 webserver.cc:460] Webserver started at http://127.10.130.193:44233/ using document root <none> and password file <none>
I20260924 07:14:20.659165 13871 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:20.659210 13871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:20.659296 13871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:20.660238 13871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "7134a40616924b4586e79293fc3709aa"
format_stamp: "Formatted at 2026-09-24 07:14:20 on dist-test-slave-7q7m"
server_key: "3c6746948c26f09d2d4e42280ff04584"
server_key_iv: "2a8d3817b62b97c0bc0c82028c782ed1"
server_key_version: "encryptionkey@0"
I20260924 07:14:20.660521 13871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "7134a40616924b4586e79293fc3709aa"
format_stamp: "Formatted at 2026-09-24 07:14:20 on dist-test-slave-7q7m"
server_key: "3c6746948c26f09d2d4e42280ff04584"
server_key_iv: "2a8d3817b62b97c0bc0c82028c782ed1"
server_key_version: "encryptionkey@0"
I20260924 07:14:20.661592 13871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:20.662158 13886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:20.662298 13871 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:20.662371 13871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "7134a40616924b4586e79293fc3709aa"
format_stamp: "Formatted at 2026-09-24 07:14:20 on dist-test-slave-7q7m"
server_key: "3c6746948c26f09d2d4e42280ff04584"
server_key_iv: "2a8d3817b62b97c0bc0c82028c782ed1"
server_key_version: "encryptionkey@0"
I20260924 07:14:20.662421 13871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:20.684757 13871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:20.686489 13871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:20.686595 13871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:20.686764 13871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:20.686990 13871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:20.687017 13871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:20.687040 13871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:20.687053 13871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:20.692196 13871 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:38245
I20260924 07:14:20.692261 13999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:38245 every 8 connection(s)
I20260924 07:14:20.692520 13871 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:14:20.695456 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 13871
I20260924 07:14:20.695554 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:20.695735 10763 external_mini_cluster.cc:1428] Setting key 164d6cbea60cdab70764680225da6fae
Sep 24 07:14:20 dist-test-slave-7q7m krb5kdc[13764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234060, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:20.701499 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:20.693130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:59193 (local address 127.10.130.254:42585)
0924 07:14:20.693183 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:20.693185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:20.693351 (+ 166us) server_negotiation.cc:407] Connection header received
0924 07:14:20.693857 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:20.693859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:20.693871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:20.693913 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:20.694794 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.695285 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.696579 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.696777 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.696831 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:20.698932 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:20.698933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:20.698933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:20.698940 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:20.700212 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:20.700661 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:20.700662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:20.700662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:20.700711 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:20.700950 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:20.700950 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:20.700950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:20.701039 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:20.701089 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:20.701378 (+ 289us) server_negotiation.cc:299] Negotiation successful
0924 07:14:20.701439 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 07:14:20.701745 14002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:20.693113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:42585 (local address 127.10.130.193:59193)
0924 07:14:20.693248 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:20.693281 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:20.693768 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:20.693991 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:20.693995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:20.694518 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:20.694701 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:20.694705 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.695378 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.695379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:20.696499 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:20.696501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.696842 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.696843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:20.696965 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:20.697376 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:20.697386 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:20.698849 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:20.700304 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:20.700310 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:20.700312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:20.700595 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:20.700767 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:20.700768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:20.700768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:20.700896 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:20.701098 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:20.701103 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:20.701257 (+ 154us) client_negotiation.cc:769] Sending connection context
0924 07:14:20.701371 (+ 114us) client_negotiation.cc:240] Negotiation successful
0924 07:14:20.701521 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260924 07:14:20.702194 14000 heartbeater.cc:347] Connected to a master server at 127.10.130.254:42585
I20260924 07:14:20.702271 14000 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:20.702423 14000 heartbeater.cc:511] Master 127.10.130.254:42585 requested a full tablet report, sending...
I20260924 07:14:20.702884 13809 ts_manager.cc:194] Registered new tserver with Master: 7134a40616924b4586e79293fc3709aa (127.10.130.193:38245)
I20260924 07:14:20.703442 13809 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:59193
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:14:20.717814 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42585
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:40627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:20.793967 14005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:20.794153 14005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:20.794174 14005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:20.795212 14005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:20.795243 14005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:20.795266 14005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:20.796620 14005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:40627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:20.796787 14005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:20.796947 14005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:20.801438 14010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:20.801446 14013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:20.801507 14011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:20.801529 14005 server_base.cc:1034] running on GCE node
I20260924 07:14:20.801935 14005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:20.802155 14005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:20.803293 14005 hybrid_clock.cc:648] HybridClock initialized: now 1790234060803277 us; error 27 us; skew 500 ppm
Sep 24 07:14:20 dist-test-slave-7q7m krb5kdc[13764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234060, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:20.805594 14005 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:14:20.805999 14005 webserver.cc:460] Webserver started at http://127.10.130.194:34999/ using document root <none> and password file <none>
I20260924 07:14:20.806190 14005 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:20.806236 14005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:20.806354 14005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:20.807333 14005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "55d974359f9e4b018e485f0a6f1d3dae"
format_stamp: "Formatted at 2026-09-24 07:14:20 on dist-test-slave-7q7m"
server_key: "35016390d424f34cbe5717ee1cd12fb0"
server_key_iv: "3d9d783c2ae072169d13bd33b2ed530d"
server_key_version: "encryptionkey@0"
I20260924 07:14:20.807652 14005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "55d974359f9e4b018e485f0a6f1d3dae"
format_stamp: "Formatted at 2026-09-24 07:14:20 on dist-test-slave-7q7m"
server_key: "35016390d424f34cbe5717ee1cd12fb0"
server_key_iv: "3d9d783c2ae072169d13bd33b2ed530d"
server_key_version: "encryptionkey@0"
I20260924 07:14:20.808849 14005 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:14:20.809478 14020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:20.809621 14005 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:20.809700 14005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "55d974359f9e4b018e485f0a6f1d3dae"
format_stamp: "Formatted at 2026-09-24 07:14:20 on dist-test-slave-7q7m"
server_key: "35016390d424f34cbe5717ee1cd12fb0"
server_key_iv: "3d9d783c2ae072169d13bd33b2ed530d"
server_key_version: "encryptionkey@0"
I20260924 07:14:20.809758 14005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:20.825577 14005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:20.827214 14005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:20.827354 14005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:20.827540 14005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:20.827817 14005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:20.827854 14005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:20.827881 14005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:20.827899 14005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:20.833087 14005 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:35909
I20260924 07:14:20.833143 14133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:35909 every 8 connection(s)
I20260924 07:14:20.833415 14005 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
Sep 24 07:14:20 dist-test-slave-7q7m krb5kdc[13764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234060, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:20.841679 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:20.834026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:47461 (local address 127.10.130.254:42585)
0924 07:14:20.834081 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:20.834083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:20.834279 (+ 196us) server_negotiation.cc:407] Connection header received
0924 07:14:20.834770 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:20.834771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:20.834780 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:20.834829 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:20.835719 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.836260 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.837254 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.837428 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.837470 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:20.839478 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:20.839480 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:20.839480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:20.839486 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:20.840582 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:20.840919 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:20.840920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:20.840920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:20.840969 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:20.841142 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:20.841142 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:20.841143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:20.841235 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:20.841278 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:20.841550 (+ 272us) server_negotiation.cc:299] Negotiation successful
0924 07:14:20.841614 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 07:14:20.841784 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 14005
I20260924 07:14:20.841904 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:20.841931 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:20.833993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:42585 (local address 127.10.130.194:47461)
0924 07:14:20.834153 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:20.834193 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:20.834691 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:20.834892 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:20.834896 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:20.835433 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:20.835631 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:20.835636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.836354 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.836355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:20.837189 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:20.837191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.837504 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.837505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:20.837632 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:20.838095 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:20.838103 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:20.839389 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:20.840651 (+ 1262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:20.840657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:20.840658 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:20.840862 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:20.841015 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:20.841015 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:20.841016 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:20.841102 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:20.841304 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:20.841310 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:20.841439 (+ 129us) client_negotiation.cc:769] Sending connection context
0924 07:14:20.841534 (+ 95us) client_negotiation.cc:240] Negotiation successful
0924 07:14:20.841706 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260924 07:14:20.842108 10763 external_mini_cluster.cc:1428] Setting key 1f2b49bafe0ed966947d3dc436fb059a
I20260924 07:14:20.842355 14134 heartbeater.cc:347] Connected to a master server at 127.10.130.254:42585
I20260924 07:14:20.842446 14134 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:20.842592 14134 heartbeater.cc:511] Master 127.10.130.254:42585 requested a full tablet report, sending...
I20260924 07:14:20.842933 13808 ts_manager.cc:194] Registered new tserver with Master: 55d974359f9e4b018e485f0a6f1d3dae (127.10.130.194:35909)
I20260924 07:14:20.843387 13808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:47461
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:14:20.863540 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42585
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:40627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:20.940153 14139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:20.940332 14139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:20.940353 14139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:20.941406 14139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:20.941440 14139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:20.941463 14139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:20.942786 14139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:40627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:20.942978 14139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:20.943164 14139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:20.947595 14144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:20.947640 14145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:20.947618 14147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:20.947750 14139 server_base.cc:1034] running on GCE node
I20260924 07:14:20.947901 14139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:20.948132 14139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:20.949292 14139 hybrid_clock.cc:648] HybridClock initialized: now 1790234060949284 us; error 30 us; skew 500 ppm
Sep 24 07:14:20 dist-test-slave-7q7m krb5kdc[13764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234060, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:20.951530 14139 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:14:20.951911 14139 webserver.cc:460] Webserver started at http://127.10.130.195:45975/ using document root <none> and password file <none>
I20260924 07:14:20.952123 14139 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:20.952167 14139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:20.952279 14139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:20.953241 14139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "9436c842e332448ca543a162162131ec"
format_stamp: "Formatted at 2026-09-24 07:14:20 on dist-test-slave-7q7m"
server_key: "bb8be51f751f5e29544d31857da0e331"
server_key_iv: "eb641aa026c41c09a66ae625613acbea"
server_key_version: "encryptionkey@0"
I20260924 07:14:20.953544 14139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "9436c842e332448ca543a162162131ec"
format_stamp: "Formatted at 2026-09-24 07:14:20 on dist-test-slave-7q7m"
server_key: "bb8be51f751f5e29544d31857da0e331"
server_key_iv: "eb641aa026c41c09a66ae625613acbea"
server_key_version: "encryptionkey@0"
I20260924 07:14:20.954658 14139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:20.955243 14154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:20.955365 14139 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:20.955443 14139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "9436c842e332448ca543a162162131ec"
format_stamp: "Formatted at 2026-09-24 07:14:20 on dist-test-slave-7q7m"
server_key: "bb8be51f751f5e29544d31857da0e331"
server_key_iv: "eb641aa026c41c09a66ae625613acbea"
server_key_version: "encryptionkey@0"
I20260924 07:14:20.955498 14139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:20.966077 14139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:20.967736 14139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:20.967857 14139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:20.968073 14139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:20.968338 14139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:20.968367 14139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:20.968395 14139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:20.968417 14139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:20.973519 14139 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:38977
I20260924 07:14:20.973577 14267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:38977 every 8 connection(s)
I20260924 07:14:20.973846 14139 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:20.977566 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 14139
I20260924 07:14:20.977643 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:20.977811 10763 external_mini_cluster.cc:1428] Setting key 91a1cf355f3574037e671baf578ac91b
Sep 24 07:14:20 dist-test-slave-7q7m krb5kdc[13764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234060, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:20.981801 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:20.974439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:59107 (local address 127.10.130.254:42585)
0924 07:14:20.974484 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:20.974486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:20.974674 (+ 188us) server_negotiation.cc:407] Connection header received
0924 07:14:20.975111 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:20.975112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:20.975120 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:20.975171 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:20.976049 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.976534 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.977492 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.977687 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.977744 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:20.979809 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:20.979811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:20.979811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:20.979818 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:20.980847 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:20.981164 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:20.981165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:20.981165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:20.981210 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:20.981368 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:20.981368 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:20.981369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:20.981456 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:20.981499 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:20.981687 (+ 188us) server_negotiation.cc:299] Negotiation successful
0924 07:14:20.981740 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 07:14:20.981982 14270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:20.974405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:42585 (local address 127.10.130.195:59107)
0924 07:14:20.974556 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:20.974603 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:20.975041 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:20.975226 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:20.975229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:20.975759 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:20.975936 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:20.975940 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.976616 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.976616 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:20.977426 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:20.977428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:20.977756 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:20.977757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:20.977890 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:20.978392 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:20.978402 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:20.979727 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:20.980913 (+ 1186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:20.980918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:20.980920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:20.981114 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:20.981249 (+ 135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:20.981250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:20.981250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:20.981331 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:20.981499 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:20.981502 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:20.981602 (+ 100us) client_negotiation.cc:769] Sending connection context
0924 07:14:20.981673 (+ 71us) client_negotiation.cc:240] Negotiation successful
0924 07:14:20.981785 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260924 07:14:20.982394 14268 heartbeater.cc:347] Connected to a master server at 127.10.130.254:42585
I20260924 07:14:20.982474 14268 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:20.982636 14268 heartbeater.cc:511] Master 127.10.130.254:42585 requested a full tablet report, sending...
I20260924 07:14:20.982962 13808 ts_manager.cc:194] Registered new tserver with Master: 9436c842e332448ca543a162162131ec (127.10.130.195:38977)
I20260924 07:14:20.983307 13808 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:59107
I20260924 07:14:20.989332 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 07:14:20 dist-test-slave-7q7m krb5kdc[13764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234060, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 24 07:14:21 dist-test-slave-7q7m krb5kdc[13764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234060, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:21.009158 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:21.003096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38974 (local address 127.10.130.254:42585)
0924 07:14:21.003157 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:21.003159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:21.003238 (+ 79us) server_negotiation.cc:407] Connection header received
0924 07:14:21.003281 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:21.003281 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:21.003290 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:21.003335 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:21.003805 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:21.004313 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:21.004967 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:21.005140 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:21.005180 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:21.007179 (+ 1999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:21.007180 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:21.007180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:21.007185 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:21.008311 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:21.008585 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:21.008586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:21.008587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:21.008637 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:21.008764 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:21.008764 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:21.008764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:21.008892 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:21.008958 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:21.009026 (+ 68us) server_negotiation.cc:299] Negotiation successful
0924 07:14:21.009077 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260924 07:14:21.009377 13808 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38974
I20260924 07:14:21.009752 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 13871
I20260924 07:14:21.013759 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 14005
I20260924 07:14:21.017411 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 14139
I20260924 07:14:21.020961 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 13777
2026-09-24T07:14:21Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1654 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:21 dist-test-slave-7q7m krb5kdc[14280](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:21 dist-test-slave-7q7m krb5kdc[14280](info): set up 2 sockets
Sep 24 07:14:21 dist-test-slave-7q7m krb5kdc[14280](info): commencing operation
krb5kdc: starting...
W20260924 07:14:21.955507 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.916s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:14:22.058754 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:46561
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:46561
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:22.135696 14293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:22.135900 14293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:22.135921 14293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:22.137096 14293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:22.137137 14293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:22.137151 14293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:22.137161 14293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:22.137173 14293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:22.138640 14293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:46561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:46561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:22.138870 14293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:22.139074 14293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:22.143581 14301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:22.143600 14298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:22.143592 14299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:22.143733 14293 server_base.cc:1034] running on GCE node
I20260924 07:14:22.144030 14293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:22.144310 14293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:22.145444 14293 hybrid_clock.cc:648] HybridClock initialized: now 1790234062145432 us; error 27 us; skew 500 ppm
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234062, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:22.151029 14293 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:14:22.151453 14293 webserver.cc:460] Webserver started at http://127.10.130.254:41063/ using document root <none> and password file <none>
I20260924 07:14:22.151661 14293 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:22.151707 14293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:22.151836 14293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:22.152940 14293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "6e4d6a557b014bb09906129c867dc71f"
format_stamp: "Formatted at 2026-09-24 07:14:22 on dist-test-slave-7q7m"
server_key: "180c71406df0f8a7d10b0d0b20c624a7"
server_key_iv: "a74ad34f110edceeb091a0a7c12fbfa5"
server_key_version: "encryptionkey@0"
I20260924 07:14:22.153270 14293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "6e4d6a557b014bb09906129c867dc71f"
format_stamp: "Formatted at 2026-09-24 07:14:22 on dist-test-slave-7q7m"
server_key: "180c71406df0f8a7d10b0d0b20c624a7"
server_key_iv: "a74ad34f110edceeb091a0a7c12fbfa5"
server_key_version: "encryptionkey@0"
I20260924 07:14:22.154492 14293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:14:22.155110 14308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:22.155272 14293 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:22.155352 14293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "6e4d6a557b014bb09906129c867dc71f"
format_stamp: "Formatted at 2026-09-24 07:14:22 on dist-test-slave-7q7m"
server_key: "180c71406df0f8a7d10b0d0b20c624a7"
server_key_iv: "a74ad34f110edceeb091a0a7c12fbfa5"
server_key_version: "encryptionkey@0"
I20260924 07:14:22.155411 14293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:22.178865 14293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:22.181001 14293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:22.181137 14293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:22.184806 14293 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:46561
I20260924 07:14:22.184857 14360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:46561 every 8 connection(s)
I20260924 07:14:22.185300 14293 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:22.185948 14361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:22.188124 14361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f: Bootstrap starting.
I20260924 07:14:22.188637 14361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:22.188867 14361 log.cc:824] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:22.189432 14361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f: No bootstrap required, opened a new log
I20260924 07:14:22.190876 14361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4d6a557b014bb09906129c867dc71f" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46561 } }
I20260924 07:14:22.191004 14361 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:22.191023 14361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e4d6a557b014bb09906129c867dc71f, State: Initialized, Role: FOLLOWER
I20260924 07:14:22.191130 14361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4d6a557b014bb09906129c867dc71f" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46561 } }
I20260924 07:14:22.191201 14361 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:22.191231 14361 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:22.191262 14361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:22.192070 14361 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4d6a557b014bb09906129c867dc71f" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46561 } }
I20260924 07:14:22.192152 14361 leader_election.cc:302] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e4d6a557b014bb09906129c867dc71f; no voters:
I20260924 07:14:22.192265 14361 leader_election.cc:288] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:22.192301 14364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:22.192466 14361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:22.192473 14364 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [term 1 LEADER]: Becoming Leader. State: Replica: 6e4d6a557b014bb09906129c867dc71f, State: Running, Role: LEADER
I20260924 07:14:22.192651 14364 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4d6a557b014bb09906129c867dc71f" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46561 } }
I20260924 07:14:22.193126 14365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e4d6a557b014bb09906129c867dc71f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4d6a557b014bb09906129c867dc71f" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46561 } } }
I20260924 07:14:22.193212 14365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:22.193415 14366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e4d6a557b014bb09906129c867dc71f. Latest consensus state: current_term: 1 leader_uuid: "6e4d6a557b014bb09906129c867dc71f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e4d6a557b014bb09906129c867dc71f" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46561 } } }
I20260924 07:14:22.193523 14366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:22.193768 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 14293
I20260924 07:14:22.193871 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:22.193922 14373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:22.194053 10763 external_mini_cluster.cc:1428] Setting key 32265b6a47dad28dfb2127210aec0e8d
I20260924 07:14:22.194535 14373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:22.196643 14373 catalog_manager.cc:1347] Generated new cluster ID: a05efd8e96c848b2bb24209e606da90e
I20260924 07:14:22.196686 14373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:22.205176 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:22.194729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53520 (local address 127.10.130.254:46561)
0924 07:14:22.194887 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:22.194897 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:22.195066 (+ 169us) server_negotiation.cc:407] Connection header received
0924 07:14:22.195270 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:22.195277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:22.195336 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:22.195442 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:22.196306 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.197739 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.198843 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.199424 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.199522 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:22.202236 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:22.202240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:22.202240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:22.202251 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:22.203786 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:22.204146 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:22.204148 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:22.204149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:22.204238 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:22.204416 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:22.204416 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:22.204417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:22.204648 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:22.204787 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:22.204890 (+ 103us) server_negotiation.cc:299] Negotiation successful
0924 07:14:22.205017 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260924 07:14:22.213618 14373 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:22.214088 14373 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:22.216957 14373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e4d6a557b014bb09906129c867dc71f: Generated new TSK 0
I20260924 07:14:22.217088 14373 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:14:22.244884 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:46561
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:22.323287 14387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:22.323468 14387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:22.323489 14387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:22.324613 14387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:22.324652 14387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:22.324680 14387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:22.326148 14387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:46561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:22.326368 14387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:22.326572 14387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:22.331137 14393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:22.331140 14392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:22.331168 14395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:22.331374 14387 server_base.cc:1034] running on GCE node
I20260924 07:14:22.331554 14387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:22.331756 14387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:22.332894 14387 hybrid_clock.cc:648] HybridClock initialized: now 1790234062332874 us; error 30 us; skew 500 ppm
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234062, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:22.335091 14387 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:14:22.335471 14387 webserver.cc:460] Webserver started at http://127.10.130.193:33401/ using document root <none> and password file <none>
I20260924 07:14:22.335664 14387 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:22.335709 14387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:22.335824 14387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:22.336833 14387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "204c24855fd24afd83fb3a30186b0675"
format_stamp: "Formatted at 2026-09-24 07:14:22 on dist-test-slave-7q7m"
server_key: "09e935b5be96f1ae2ed4548f1c90874c"
server_key_iv: "c833b7b8e27eb2767bbc21d97c38d2a1"
server_key_version: "encryptionkey@0"
I20260924 07:14:22.337131 14387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "204c24855fd24afd83fb3a30186b0675"
format_stamp: "Formatted at 2026-09-24 07:14:22 on dist-test-slave-7q7m"
server_key: "09e935b5be96f1ae2ed4548f1c90874c"
server_key_iv: "c833b7b8e27eb2767bbc21d97c38d2a1"
server_key_version: "encryptionkey@0"
I20260924 07:14:22.338284 14387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:22.338904 14402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:22.339040 14387 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:22.339116 14387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "204c24855fd24afd83fb3a30186b0675"
format_stamp: "Formatted at 2026-09-24 07:14:22 on dist-test-slave-7q7m"
server_key: "09e935b5be96f1ae2ed4548f1c90874c"
server_key_iv: "c833b7b8e27eb2767bbc21d97c38d2a1"
server_key_version: "encryptionkey@0"
I20260924 07:14:22.339178 14387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:22.348148 14387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:22.349650 14387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:22.349781 14387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:22.349960 14387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:22.350215 14387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:22.350247 14387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:22.350273 14387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:22.350291 14387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:22.355360 14387 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:45315
I20260924 07:14:22.355417 14515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:45315 every 8 connection(s)
I20260924 07:14:22.355651 14387 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:14:22.359344 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 14387
I20260924 07:14:22.359436 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:22.359608 10763 external_mini_cluster.cc:1428] Setting key 23c31f9f94bcdb8404fe7ea536baad66
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234062, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:22.364771 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:22.356286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:40809 (local address 127.10.130.254:46561)
0924 07:14:22.356339 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:22.356341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:22.356527 (+ 186us) server_negotiation.cc:407] Connection header received
0924 07:14:22.357336 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:22.357336 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:22.357345 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:22.357411 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:22.357945 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.358429 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.359563 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.359849 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.359892 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:22.362328 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:22.362330 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:22.362330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:22.362336 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:22.363570 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:22.363923 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:22.363924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:22.363924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:22.364001 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:22.364206 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:22.364207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:22.364207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:22.364322 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:22.364375 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:22.364646 (+ 271us) server_negotiation.cc:299] Negotiation successful
0924 07:14:22.364710 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 07:14:22.364980 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:22.356271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:46561 (local address 127.10.130.193:40809)
0924 07:14:22.356415 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:22.356453 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:22.357262 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:22.357461 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:22.357463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:22.357724 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:22.357890 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:22.357894 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.358493 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.358494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:22.359464 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:22.359467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.360019 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.360021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:22.360159 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:22.360566 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:22.360574 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:22.362241 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:22.363656 (+ 1415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:22.363662 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:22.363663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:22.363865 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:22.364065 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:22.364065 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:22.364065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:22.364153 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:22.364379 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:22.364384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:22.364529 (+ 145us) client_negotiation.cc:769] Sending connection context
0924 07:14:22.364632 (+ 103us) client_negotiation.cc:240] Negotiation successful
0924 07:14:22.364782 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
I20260924 07:14:22.365422 14516 heartbeater.cc:347] Connected to a master server at 127.10.130.254:46561
I20260924 07:14:22.365502 14516 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:22.365653 14516 heartbeater.cc:511] Master 127.10.130.254:46561 requested a full tablet report, sending...
I20260924 07:14:22.366086 14325 ts_manager.cc:194] Registered new tserver with Master: 204c24855fd24afd83fb3a30186b0675 (127.10.130.193:45315)
I20260924 07:14:22.366648 14325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:40809
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:14:22.382157 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:46561
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:22.460556 14521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:22.460748 14521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:22.460770 14521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:22.461861 14521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:22.461896 14521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:22.461920 14521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:22.463323 14521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:46561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:22.463502 14521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:22.463672 14521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:22.468080 14529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:22.468147 14527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:22.468080 14526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:22.468430 14521 server_base.cc:1034] running on GCE node
I20260924 07:14:22.468664 14521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:22.468919 14521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:22.470088 14521 hybrid_clock.cc:648] HybridClock initialized: now 1790234062470068 us; error 28 us; skew 500 ppm
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234062, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:22.472513 14521 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:14:22.473016 14521 webserver.cc:460] Webserver started at http://127.10.130.194:41417/ using document root <none> and password file <none>
I20260924 07:14:22.473214 14521 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:22.473253 14521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:22.473340 14521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:22.474262 14521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "52e83cdacb564ae192f41fb5d3f272a2"
format_stamp: "Formatted at 2026-09-24 07:14:22 on dist-test-slave-7q7m"
server_key: "7716e2bf34ca8438402f861067fc6a4c"
server_key_iv: "db43b01f3d1c8c76f14693a0047de1c2"
server_key_version: "encryptionkey@0"
I20260924 07:14:22.474525 14521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "52e83cdacb564ae192f41fb5d3f272a2"
format_stamp: "Formatted at 2026-09-24 07:14:22 on dist-test-slave-7q7m"
server_key: "7716e2bf34ca8438402f861067fc6a4c"
server_key_iv: "db43b01f3d1c8c76f14693a0047de1c2"
server_key_version: "encryptionkey@0"
I20260924 07:14:22.475630 14521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 07:14:22.476250 14536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:22.476379 14521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:22.476455 14521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "52e83cdacb564ae192f41fb5d3f272a2"
format_stamp: "Formatted at 2026-09-24 07:14:22 on dist-test-slave-7q7m"
server_key: "7716e2bf34ca8438402f861067fc6a4c"
server_key_iv: "db43b01f3d1c8c76f14693a0047de1c2"
server_key_version: "encryptionkey@0"
I20260924 07:14:22.476511 14521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:22.486521 14521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:22.488102 14521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:22.488222 14521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:22.488404 14521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:22.488672 14521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:22.488703 14521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:22.488730 14521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:22.488754 14521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:22.494061 14521 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:38017
I20260924 07:14:22.494112 14649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:38017 every 8 connection(s)
I20260924 07:14:22.494387 14521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:14:22.495828 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 14521
I20260924 07:14:22.495911 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:22.496104 10763 external_mini_cluster.cc:1428] Setting key 5d3cc8951ee0ae126a05ac3a4dd64066
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234062, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
I20260924 07:14:22.503602 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:22.494974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:39051 (local address 127.10.130.254:46561)
0924 07:14:22.495014 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:22.495016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:22.495212 (+ 196us) server_negotiation.cc:407] Connection header received
0924 07:14:22.496196 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:22.496198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:22.496210 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:22.496255 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:22.496911 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.497394 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.498697 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.498941 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.498999 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:22.501139 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:22.501140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:22.501141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:22.501147 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:22.502325 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:22.502735 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:22.502736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:22.502736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:22.502780 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:22.503019 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:22.503020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:22.503020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:22.503125 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:22.503174 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:22.503471 (+ 297us) server_negotiation.cc:299] Negotiation successful
0924 07:14:22.503539 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:14:22.503786 14652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:22.494949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:46561 (local address 127.10.130.194:39051)
0924 07:14:22.495108 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:22.495140 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:22.496106 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:22.496323 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:22.496327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:22.496653 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:22.496837 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:22.496842 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.497466 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.497467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:22.498631 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:22.498633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.499028 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.499029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:22.499138 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:22.499582 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:22.499590 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:22.501043 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:22.502397 (+ 1354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:22.502403 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:22.502405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:22.502669 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:22.502832 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:22.502833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:22.502833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:22.502952 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:22.503180 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:22.503185 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:22.503331 (+ 146us) client_negotiation.cc:769] Sending connection context
0924 07:14:22.503441 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 07:14:22.503592 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260924 07:14:22.504232 14650 heartbeater.cc:347] Connected to a master server at 127.10.130.254:46561
I20260924 07:14:22.504318 14650 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:22.504472 14650 heartbeater.cc:511] Master 127.10.130.254:46561 requested a full tablet report, sending...
I20260924 07:14:22.504796 14325 ts_manager.cc:194] Registered new tserver with Master: 52e83cdacb564ae192f41fb5d3f272a2 (127.10.130.194:38017)
I20260924 07:14:22.505148 14325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:39051
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:14:22.517814 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:46561
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:22.594086 14655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:22.594273 14655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:22.594295 14655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:22.595399 14655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:22.595440 14655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:22.595468 14655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:22.596946 14655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:46561
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:22.597149 14655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:22.597344 14655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:22.601858 14663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:22.601886 14660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:22.601898 14661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:22.601967 14655 server_base.cc:1034] running on GCE node
I20260924 07:14:22.602255 14655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:22.602473 14655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:22.603609 14655 hybrid_clock.cc:648] HybridClock initialized: now 1790234062603587 us; error 27 us; skew 500 ppm
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234062, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:22.605892 14655 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:14:22.606287 14655 webserver.cc:460] Webserver started at http://127.10.130.195:35595/ using document root <none> and password file <none>
I20260924 07:14:22.606453 14655 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:22.606490 14655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:22.606580 14655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:22.607522 14655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "7152e633ff4c4013b7f5d7f488c7968f"
format_stamp: "Formatted at 2026-09-24 07:14:22 on dist-test-slave-7q7m"
server_key: "5b7e13446d70590f4067ede22b63110b"
server_key_iv: "7f0dc805b0b003b7589c4665ca850dad"
server_key_version: "encryptionkey@0"
I20260924 07:14:22.607810 14655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "7152e633ff4c4013b7f5d7f488c7968f"
format_stamp: "Formatted at 2026-09-24 07:14:22 on dist-test-slave-7q7m"
server_key: "5b7e13446d70590f4067ede22b63110b"
server_key_iv: "7f0dc805b0b003b7589c4665ca850dad"
server_key_version: "encryptionkey@0"
I20260924 07:14:22.608958 14655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:14:22.609543 14670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:22.609692 14655 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:22.609771 14655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "7152e633ff4c4013b7f5d7f488c7968f"
format_stamp: "Formatted at 2026-09-24 07:14:22 on dist-test-slave-7q7m"
server_key: "5b7e13446d70590f4067ede22b63110b"
server_key_iv: "7f0dc805b0b003b7589c4665ca850dad"
server_key_version: "encryptionkey@0"
I20260924 07:14:22.609831 14655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:22.619537 14655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:22.621497 14655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:22.621613 14655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:22.621793 14655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:22.622040 14655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:22.622071 14655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:22.622097 14655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:22.622119 14655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:22.627242 14655 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:42267
I20260924 07:14:22.627300 14783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:42267 every 8 connection(s)
I20260924 07:14:22.627558 14655 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:22.631479 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 14655
I20260924 07:14:22.631572 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:22.631757 10763 external_mini_cluster.cc:1428] Setting key 7154396e475a73256a4dc7c801493b21
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234062, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:22.635692 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:22.628187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:56069 (local address 127.10.130.254:46561)
0924 07:14:22.628240 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:22.628242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:22.628432 (+ 190us) server_negotiation.cc:407] Connection header received
0924 07:14:22.629266 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:22.629267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:22.629277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:22.629321 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:22.629926 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.630403 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.631296 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.631476 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.631532 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:22.633627 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:22.633629 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:22.633629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:22.633636 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:22.634650 (+ 1014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:22.634971 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:22.634972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:22.634972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:22.635014 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:22.635176 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:22.635177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:22.635177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:22.635265 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:22.635309 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:22.635549 (+ 240us) server_negotiation.cc:299] Negotiation successful
0924 07:14:22.635631 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 07:14:22.635725 14786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:22.628168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:46561 (local address 127.10.130.195:56069)
0924 07:14:22.628324 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:22.628357 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:22.629186 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:22.629380 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:22.629383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:22.629604 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:22.629787 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:22.629791 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.630472 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.630473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:22.631228 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:22.631230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.631556 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.631557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:22.631700 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:22.632213 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:22.632222 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:22.633533 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:22.634717 (+ 1184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:22.634723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:22.634724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:22.634922 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:22.635056 (+ 134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:22.635056 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:22.635056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:22.635137 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:22.635305 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:22.635308 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:22.635407 (+ 99us) client_negotiation.cc:769] Sending connection context
0924 07:14:22.635479 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 07:14:22.635583 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260924 07:14:22.636142 14784 heartbeater.cc:347] Connected to a master server at 127.10.130.254:46561
I20260924 07:14:22.636214 14784 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:22.636373 14784 heartbeater.cc:511] Master 127.10.130.254:46561 requested a full tablet report, sending...
I20260924 07:14:22.636682 14325 ts_manager.cc:194] Registered new tserver with Master: 7152e633ff4c4013b7f5d7f488c7968f (127.10.130.195:42267)
I20260924 07:14:22.637033 14325 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:56069
I20260924 07:14:22.643289 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.193@KRBTEST.COM
I20260924 07:14:22.663275 14791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:22.656579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35140 (local address 127.10.130.193:45315)
0924 07:14:22.656713 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:22.656717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:22.656729 (+ 12us) server_negotiation.cc:407] Connection header received
0924 07:14:22.656752 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:22.656754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:22.656767 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:22.656827 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:22.657313 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.658039 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.658750 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.659049 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.659105 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:22.661061 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:22.661068 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:22.661068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:22.661075 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:22.662267 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:22.662548 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:22.662551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:22.662552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:22.662605 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:22.662764 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:22.662764 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:22.662765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:22.662887 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:22.663011 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:22.663092 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 07:14:22.663171 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234062, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:22.670094 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:22.664364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53524 (local address 127.10.130.254:46561)
0924 07:14:22.664419 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:22.664421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:22.664431 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:14:22.664495 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:22.664496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:22.664504 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:22.664538 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:22.665069 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.665533 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.666169 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:22.666341 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:22.666383 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:22.668254 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:22.668256 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:22.668256 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:22.668263 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:22.669264 (+ 1001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:22.669531 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:22.669532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:22.669532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:22.669574 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:22.669702 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:22.669703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:22.669703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:22.669825 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:22.669889 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:22.669966 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 07:14:22.670021 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260924 07:14:22.670315 14325 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53524
I20260924 07:14:22.670625 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 14387
I20260924 07:14:22.674408 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 14521
I20260924 07:14:22.677769 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 14655
I20260924 07:14:22.681169 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 14293
2026-09-24T07:14:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1659 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14796](info): set up 2 sockets
Sep 24 07:14:22 dist-test-slave-7q7m krb5kdc[14796](info): commencing operation
krb5kdc: starting...
W20260924 07:14:23.631553 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.932s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:23 dist-test-slave-7q7m krb5kdc[14796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:14:23.739002 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:43513
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:43513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:23.815688 14809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:23.815891 14809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:23.815920 14809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:23.817080 14809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:23.817121 14809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:23.817139 14809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:23.817154 14809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:23.817169 14809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:23.818612 14809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:43513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:43513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:23.818836 14809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:23.819029 14809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:23.823334 14815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:23.823369 14814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:23.823377 14817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:23.823467 14809 server_base.cc:1034] running on GCE node
I20260924 07:14:23.823673 14809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:23.823926 14809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:23.825059 14809 hybrid_clock.cc:648] HybridClock initialized: now 1790234063825047 us; error 33 us; skew 500 ppm
Sep 24 07:14:23 dist-test-slave-7q7m krb5kdc[14796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234063, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:23.827534 14809 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:14:23.827827 14809 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:23.827872 14809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:23.828002 14809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:23.828998 14809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "aacd0099ec4140ca88275c5031d505d1"
format_stamp: "Formatted at 2026-09-24 07:14:23 on dist-test-slave-7q7m"
server_key: "4dfc30c124947b436924ff58c498c955"
server_key_iv: "c56e6dd0f5a7714a9f1c09bd39342b55"
server_key_version: "encryptionkey@0"
I20260924 07:14:23.829312 14809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "aacd0099ec4140ca88275c5031d505d1"
format_stamp: "Formatted at 2026-09-24 07:14:23 on dist-test-slave-7q7m"
server_key: "4dfc30c124947b436924ff58c498c955"
server_key_iv: "c56e6dd0f5a7714a9f1c09bd39342b55"
server_key_version: "encryptionkey@0"
I20260924 07:14:23.830477 14809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:23.831137 14823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:23.831298 14809 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:23.831377 14809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "aacd0099ec4140ca88275c5031d505d1"
format_stamp: "Formatted at 2026-09-24 07:14:23 on dist-test-slave-7q7m"
server_key: "4dfc30c124947b436924ff58c498c955"
server_key_iv: "c56e6dd0f5a7714a9f1c09bd39342b55"
server_key_version: "encryptionkey@0"
I20260924 07:14:23.831434 14809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:23.849821 14809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:23.851485 14809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:23.851616 14809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:23.855144 14809 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:43513
I20260924 07:14:23.855183 14875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:43513 every 8 connection(s)
I20260924 07:14:23.855424 14809 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:23.856307 14876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:23.858420 14876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1: Bootstrap starting.
I20260924 07:14:23.858927 14876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:23.859148 14876 log.cc:824] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:23.859721 14876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1: No bootstrap required, opened a new log
I20260924 07:14:23.861164 14876 raft_consensus.cc:348] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacd0099ec4140ca88275c5031d505d1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43513 } }
I20260924 07:14:23.861295 14876 raft_consensus.cc:374] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:23.861325 14876 raft_consensus.cc:729] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aacd0099ec4140ca88275c5031d505d1, State: Initialized, Role: FOLLOWER
I20260924 07:14:23.861433 14876 consensus_queue.cc:260] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacd0099ec4140ca88275c5031d505d1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43513 } }
I20260924 07:14:23.861501 14876 raft_consensus.cc:388] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:23.861527 14876 raft_consensus.cc:482] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:23.861557 14876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:23.862255 14876 raft_consensus.cc:504] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacd0099ec4140ca88275c5031d505d1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43513 } }
I20260924 07:14:23.862335 14876 leader_election.cc:302] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aacd0099ec4140ca88275c5031d505d1; no voters:
I20260924 07:14:23.862453 14876 leader_election.cc:288] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:23.862497 14879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:23.862637 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 14809
I20260924 07:14:23.862676 14879 raft_consensus.cc:686] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [term 1 LEADER]: Becoming Leader. State: Replica: aacd0099ec4140ca88275c5031d505d1, State: Running, Role: LEADER
I20260924 07:14:23.862699 14876 sys_catalog.cc:564] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:23.862735 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:23.862757 14879 consensus_queue.cc:237] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacd0099ec4140ca88275c5031d505d1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43513 } }
I20260924 07:14:23.862931 10763 external_mini_cluster.cc:1428] Setting key 67d61aeb0ebe5169430ed572eeb2e37f
I20260924 07:14:23.863049 14880 sys_catalog.cc:455] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aacd0099ec4140ca88275c5031d505d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacd0099ec4140ca88275c5031d505d1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43513 } } }
I20260924 07:14:23.863111 14880 sys_catalog.cc:458] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:23.863256 14881 sys_catalog.cc:455] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aacd0099ec4140ca88275c5031d505d1. Latest consensus state: current_term: 1 leader_uuid: "aacd0099ec4140ca88275c5031d505d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aacd0099ec4140ca88275c5031d505d1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43513 } } }
I20260924 07:14:23.863307 14881 sys_catalog.cc:458] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:23.863785 14885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:23.864437 14885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:23.866662 14885 catalog_manager.cc:1347] Generated new cluster ID: c1cc5dbd6b704d5c819b1d10d7360ed4
I20260924 07:14:23.866714 14885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:14:23 dist-test-slave-7q7m krb5kdc[14796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:23.872880 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:23.863544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37410 (local address 127.10.130.254:43513)
0924 07:14:23.863811 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:23.863820 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:23.863855 (+ 35us) server_negotiation.cc:407] Connection header received
0924 07:14:23.863972 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:23.864009 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:23.864067 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:23.864201 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:23.864914 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:23.866227 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:23.867051 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:23.867481 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:23.867571 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:23.869654 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:23.869657 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:23.869658 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:23.869667 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:23.871549 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:23.871850 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:23.871853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:23.871853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:23.871928 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:23.872113 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:23.872113 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:23.872114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:23.872336 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:23.872486 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:23.872562 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 07:14:23.872706 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":84,"threads_started":1}
I20260924 07:14:23.887400 14885 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:23.887912 14885 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:23.891647 14885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1: Generated new TSK 0
I20260924 07:14:23.891803 14885 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:14:23.916522 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43513
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:24.007071 14902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:24.007272 14902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:24.007319 14902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:24.008498 14902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:24.008540 14902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:24.008575 14902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:24.010079 14902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:24.010305 14902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:24.010501 14902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:24.015002 14907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:24.015030 14908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:24.015030 14910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:24.015125 14902 server_base.cc:1034] running on GCE node
I20260924 07:14:24.015368 14902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:24.015611 14902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:24.016743 14902 hybrid_clock.cc:648] HybridClock initialized: now 1790234064016722 us; error 30 us; skew 500 ppm
Sep 24 07:14:24 dist-test-slave-7q7m krb5kdc[14796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234064, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:24.018929 14902 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:14:24.019263 14902 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:24.019315 14902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:24.019433 14902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:24.020502 14902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "37e5ba8a90b347ce9fcb51fce3fada8e"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "a15d0a5db4537c6f55ac0c5c87e37b40"
server_key_iv: "de74d1d5a3e2f0c6b359a3f678defac8"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.020815 14902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "37e5ba8a90b347ce9fcb51fce3fada8e"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "a15d0a5db4537c6f55ac0c5c87e37b40"
server_key_iv: "de74d1d5a3e2f0c6b359a3f678defac8"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.021934 14902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:24.022521 14916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.022663 14902 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:24.022733 14902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "37e5ba8a90b347ce9fcb51fce3fada8e"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "a15d0a5db4537c6f55ac0c5c87e37b40"
server_key_iv: "de74d1d5a3e2f0c6b359a3f678defac8"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.022790 14902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:24.039681 14902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:24.041572 14902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:24.041687 14902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:24.041898 14902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:24.042160 14902 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:24.042191 14902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.042212 14902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:24.042233 14902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.047397 14902 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:39939
I20260924 07:14:24.047451 15029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:39939 every 8 connection(s)
I20260924 07:14:24.047672 14902 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:14:24.051098 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 14902
I20260924 07:14:24.051186 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:24.051349 10763 external_mini_cluster.cc:1428] Setting key 8b7720779e7956457f862676adc9516a
Sep 24 07:14:24 dist-test-slave-7q7m krb5kdc[14796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234064, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:24.056456 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.048320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:52563 (local address 127.10.130.254:43513)
0924 07:14:24.048383 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:24.048385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:24.048566 (+ 181us) server_negotiation.cc:407] Connection header received
0924 07:14:24.049048 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:24.049050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:24.049061 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:24.049140 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:24.050023 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.050517 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.051638 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.051867 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.051929 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.053908 (+ 1979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:24.053910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:24.053910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:24.053916 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:24.055148 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:24.055583 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:24.055583 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:24.055584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:24.055627 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:24.055873 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:24.055873 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:24.055873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:24.055960 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:24.056039 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:24.056326 (+ 287us) server_negotiation.cc:299] Negotiation successful
0924 07:14:24.056393 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 07:14:24.056689 15032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.048297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:43513 (local address 127.10.130.193:52563)
0924 07:14:24.048457 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:24.048491 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:24.048953 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:24.049221 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:24.049226 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:24.049463 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:24.049929 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.049934 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.050601 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.050602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.051522 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.051525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.051942 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.051943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.052091 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.052510 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:24.052519 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:24.053827 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:24.055220 (+ 1393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:24.055228 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:24.055230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:24.055510 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:24.055678 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:24.055678 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:24.055679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:24.055817 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:24.056031 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:24.056037 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:24.056196 (+ 159us) client_negotiation.cc:769] Sending connection context
0924 07:14:24.056315 (+ 119us) client_negotiation.cc:240] Negotiation successful
0924 07:14:24.056470 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260924 07:14:24.057144 15030 heartbeater.cc:347] Connected to a master server at 127.10.130.254:43513
I20260924 07:14:24.057224 15030 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:24.057353 15030 heartbeater.cc:511] Master 127.10.130.254:43513 requested a full tablet report, sending...
I20260924 07:14:24.057821 14840 ts_manager.cc:194] Registered new tserver with Master: 37e5ba8a90b347ce9fcb51fce3fada8e (127.10.130.193:39939)
I20260924 07:14:24.058389 14840 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:52563
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:14:24.072224 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43513
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:24.149025 15035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:24.149204 15035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:24.149225 15035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:24.150296 15035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:24.150326 15035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:24.150352 15035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:24.151749 15035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:24.151922 15035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:24.152128 15035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:24.156572 15040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:24.156661 15035 server_base.cc:1034] running on GCE node
W20260924 07:14:24.156641 15041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:24.156607 15043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:24.156985 15035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:24.157199 15035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:24.158329 15035 hybrid_clock.cc:648] HybridClock initialized: now 1790234064158316 us; error 28 us; skew 500 ppm
Sep 24 07:14:24 dist-test-slave-7q7m krb5kdc[14796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234064, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:24.160571 15035 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:14:24.160856 15035 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:24.160900 15035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:24.161015 15035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:24.161942 15035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "5e7c002de03149f6939c8b6df41bbac9"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "8cd6e4b1c3367ba8b4a2118d46b7d1aa"
server_key_iv: "9d51491c48f0f2a1c26af7243785b829"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.162273 15035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "5e7c002de03149f6939c8b6df41bbac9"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "8cd6e4b1c3367ba8b4a2118d46b7d1aa"
server_key_iv: "9d51491c48f0f2a1c26af7243785b829"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.163440 15035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:24.164077 15049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.164211 15035 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.164287 15035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "5e7c002de03149f6939c8b6df41bbac9"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "8cd6e4b1c3367ba8b4a2118d46b7d1aa"
server_key_iv: "9d51491c48f0f2a1c26af7243785b829"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.164348 15035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:24.170668 15035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:24.172549 15035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:24.172667 15035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:24.172848 15035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:24.173105 15035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:24.173135 15035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.173161 15035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:24.173179 15035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.178282 15035 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:46809
I20260924 07:14:24.178330 15162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:46809 every 8 connection(s)
I20260924 07:14:24.178546 15035 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
Sep 24 07:14:24 dist-test-slave-7q7m krb5kdc[14796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234064, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:24.186034 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 15035
I20260924 07:14:24.186117 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:24.186291 10763 external_mini_cluster.cc:1428] Setting key a6fcce9be91c51829e883ba76c9dfb80
I20260924 07:14:24.186559 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.179133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:37593 (local address 127.10.130.254:43513)
0924 07:14:24.179176 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:24.179178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:24.179340 (+ 162us) server_negotiation.cc:407] Connection header received
0924 07:14:24.179796 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:24.179797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:24.179809 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:24.179865 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:24.180727 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.181222 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.182159 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.182359 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.182412 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.184216 (+ 1804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:24.184217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:24.184218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:24.184224 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:24.185378 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:24.185710 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:24.185711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:24.185712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:24.185760 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:24.185959 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:24.185960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:24.185960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:24.186059 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:24.186112 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:24.186390 (+ 278us) server_negotiation.cc:299] Negotiation successful
0924 07:14:24.186471 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:14:24.186671 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.179095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:43513 (local address 127.10.130.194:37593)
0924 07:14:24.179225 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:24.179265 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:24.179702 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:24.179932 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:24.179936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:24.180189 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:24.180634 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.180638 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.181297 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.181298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.182069 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.182071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.182420 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.182421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.182537 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.182932 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:24.182939 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:24.184130 (+ 1191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:24.185448 (+ 1318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:24.185454 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:24.185455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:24.185647 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:24.185810 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:24.185810 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:24.185811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:24.185906 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:24.186115 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:24.186121 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:24.186257 (+ 136us) client_negotiation.cc:769] Sending connection context
0924 07:14:24.186340 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 07:14:24.186475 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":64,"threads_started":1}
I20260924 07:14:24.187071 15163 heartbeater.cc:347] Connected to a master server at 127.10.130.254:43513
I20260924 07:14:24.187155 15163 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:24.187284 15163 heartbeater.cc:511] Master 127.10.130.254:43513 requested a full tablet report, sending...
I20260924 07:14:24.187621 14840 ts_manager.cc:194] Registered new tserver with Master: 5e7c002de03149f6939c8b6df41bbac9 (127.10.130.194:46809)
I20260924 07:14:24.188076 14840 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:37593
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:14:24.206590 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43513
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:37977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:24.282069 15168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:24.282253 15168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:24.282274 15168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:24.283385 15168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:24.283422 15168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:24.283450 15168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:24.284915 15168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:37977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43513
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:24.285122 15168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:24.285308 15168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 07:14:24.289780 15168 server_base.cc:1034] running on GCE node
W20260924 07:14:24.289734 15174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:24.289716 15173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:24.289883 15176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:24.290164 15168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:24.290385 15168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:24.291519 15168 hybrid_clock.cc:648] HybridClock initialized: now 1790234064291507 us; error 25 us; skew 500 ppm
Sep 24 07:14:24 dist-test-slave-7q7m krb5kdc[14796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234064, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:24.293761 15168 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:14:24.294019 15168 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:24.294059 15168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:24.294143 15168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:24.295035 15168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "53289899844449b2b4e13b9e84a75219"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "d67138e0dcc50d1cd6edff44f9287318"
server_key_iv: "0bcb74e6fb931b98393d7fdd8fae44a0"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.295305 15168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "53289899844449b2b4e13b9e84a75219"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "d67138e0dcc50d1cd6edff44f9287318"
server_key_iv: "0bcb74e6fb931b98393d7fdd8fae44a0"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.296406 15168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260924 07:14:24.296990 15182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.297124 15168 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.297191 15168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "53289899844449b2b4e13b9e84a75219"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "d67138e0dcc50d1cd6edff44f9287318"
server_key_iv: "0bcb74e6fb931b98393d7fdd8fae44a0"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.297254 15168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:24.316587 15168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:24.318472 15168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:24.318593 15168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:24.318825 15168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:24.319115 15168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:24.319149 15168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.319169 15168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:24.319192 15168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.324508 15168 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:45667
I20260924 07:14:24.324558 15295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:45667 every 8 connection(s)
I20260924 07:14:24.324790 15168 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:24.330277 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 15168
I20260924 07:14:24.330379 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
Sep 24 07:14:24 dist-test-slave-7q7m krb5kdc[14796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234064, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:24.330561 10763 external_mini_cluster.cc:1428] Setting key fc5b12caf6ef2736fcc7d56ed3025932
I20260924 07:14:24.333425 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.325456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:54919 (local address 127.10.130.254:43513)
0924 07:14:24.325500 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:24.325502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:24.325692 (+ 190us) server_negotiation.cc:407] Connection header received
0924 07:14:24.326135 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:24.326136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:24.326149 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:24.326204 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:24.327167 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.327677 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.328658 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.328862 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.328919 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.330909 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:24.330911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:24.330911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:24.330918 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:24.332204 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:24.332579 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:24.332580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:24.332580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:24.332634 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:24.332847 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:24.332848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:24.332849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:24.332959 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:24.333013 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:24.333262 (+ 249us) server_negotiation.cc:299] Negotiation successful
0924 07:14:24.333338 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:14:24.333634 15298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.325436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:43513 (local address 127.10.130.195:54919)
0924 07:14:24.325573 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:24.325609 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:24.326056 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:24.326266 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:24.326270 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:24.326613 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:24.327088 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.327093 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.327749 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.327750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.328582 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.328585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.328926 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.328927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.329023 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.329429 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:24.329437 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:24.330828 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:24.332274 (+ 1446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:24.332280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:24.332282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:24.332482 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:24.332689 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:24.332690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:24.332690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:24.332792 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:24.333011 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:24.333016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:24.333144 (+ 128us) client_negotiation.cc:769] Sending connection context
0924 07:14:24.333229 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 07:14:24.333381 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260924 07:14:24.334038 15296 heartbeater.cc:347] Connected to a master server at 127.10.130.254:43513
I20260924 07:14:24.334118 15296 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:24.334249 15296 heartbeater.cc:511] Master 127.10.130.254:43513 requested a full tablet report, sending...
I20260924 07:14:24.334535 14840 ts_manager.cc:194] Registered new tserver with Master: 53289899844449b2b4e13b9e84a75219 (127.10.130.195:45667)
I20260924 07:14:24.334921 14840 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:54919
I20260924 07:14:24.342099 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:14:24.348140 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.343270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.10.130.254:43513)
0924 07:14:24.343325 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:24.343327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:24.343404 (+ 77us) server_negotiation.cc:407] Connection header received
0924 07:14:24.343440 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:24.343441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:24.343449 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:24.343490 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:24.344058 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.344507 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.345174 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.345343 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.345383 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.346333 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:24.346334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:24.346334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:24.346338 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:24.347319 (+ 981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:24.347583 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:24.347583 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:24.347583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:24.347626 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:24.347738 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:24.347739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:24.347739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:24.347858 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:24.347924 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:24.348007 (+ 83us) server_negotiation.cc:299] Negotiation successful
0924 07:14:24.348076 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 07:14:24.349488 14840 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:14:24.350936 14840 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:24.356900 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.352802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:45667 (local address 127.0.0.1:53914)
0924 07:14:24.353108 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:24.353142 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:24.353204 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:24.353842 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:24.353843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:24.353845 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:24.353885 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.353886 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.354683 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.354684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.355871 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.355874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.356710 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.356710 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.356763 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.356767 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:24.356812 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 07:14:24.356845 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":62,"threads_started":1}
I20260924 07:14:24.356925 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.353111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54212 (local address 127.10.130.193:39939)
0924 07:14:24.353289 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:24.353294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:24.353303 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:24.353444 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:24.353446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:24.353462 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:24.353530 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:24.353749 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.354816 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.355806 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.356527 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.356594 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.356645 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:24.356807 (+ 162us) server_negotiation.cc:299] Negotiation successful
0924 07:14:24.356848 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":71,"threads_started":1}
I20260924 07:14:24.356994 15306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.352919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49576 (local address 127.10.130.194:46809)
0924 07:14:24.353373 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:24.353375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:24.353382 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:24.353417 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:24.353419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:24.353435 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:24.353517 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:24.353670 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.354780 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.356009 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.356490 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.356555 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.356609 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:24.356877 (+ 268us) server_negotiation.cc:299] Negotiation successful
0924 07:14:24.356920 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":89,"threads_started":1}
I20260924 07:14:24.356959 15304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.352984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:39939 (local address 127.0.0.1:54212)
0924 07:14:24.353165 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:24.353173 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:24.353214 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:24.353631 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:24.353632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:24.353634 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:24.353672 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.353673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.354909 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.354910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.355738 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.355741 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.356679 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.356680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.356738 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.356742 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:24.356790 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 07:14:24.356831 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":68,"threads_started":1}
I20260924 07:14:24.356838 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.352822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:46809 (local address 127.0.0.1:49576)
0924 07:14:24.353109 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:24.353143 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:24.353207 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:24.353529 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:24.353532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:24.353534 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:24.353592 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.353594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.354873 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.354874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.355936 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.355938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.356562 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.356563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.356636 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.356648 (+ 12us) client_negotiation.cc:769] Sending connection context
0924 07:14:24.356710 (+ 62us) client_negotiation.cc:240] Negotiation successful
0924 07:14:24.356767 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":43,"threads_started":1}
I20260924 07:14:24.357446 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.352959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53914 (local address 127.10.130.195:45667)
0924 07:14:24.353640 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:24.353644 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:24.353653 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:24.353686 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:24.353688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:24.353700 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:24.353765 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:24.353948 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.354592 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.355955 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.356641 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.356703 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.356747 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:24.357325 (+ 578us) server_negotiation.cc:299] Negotiation successful
0924 07:14:24.357370 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":58,"threads_started":1}
I20260924 07:14:24.357955 14964 tablet_service.cc:1467] Processing CreateTablet for tablet 74cb56098e77413fa4d99658245afb92 (DEFAULT_TABLE table=test-table [id=c192204c572f4e57be5c4baf23aaf0a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:24.357993 15097 tablet_service.cc:1467] Processing CreateTablet for tablet 74cb56098e77413fa4d99658245afb92 (DEFAULT_TABLE table=test-table [id=c192204c572f4e57be5c4baf23aaf0a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:24.358269 15097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74cb56098e77413fa4d99658245afb92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:24.358274 14964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74cb56098e77413fa4d99658245afb92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:24.360391 15230 tablet_service.cc:1467] Processing CreateTablet for tablet 74cb56098e77413fa4d99658245afb92 (DEFAULT_TABLE table=test-table [id=c192204c572f4e57be5c4baf23aaf0a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:24.360692 15230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74cb56098e77413fa4d99658245afb92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:24.360875 15310 tablet_bootstrap.cc:492] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9: Bootstrap starting.
I20260924 07:14:24.361346 15310 tablet_bootstrap.cc:654] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:24.361797 15310 log.cc:824] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:24.362082 15311 tablet_bootstrap.cc:492] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e: Bootstrap starting.
I20260924 07:14:24.362471 15310 tablet_bootstrap.cc:492] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9: No bootstrap required, opened a new log
I20260924 07:14:24.362526 15310 ts_tablet_manager.cc:1397] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:24.362545 15311 tablet_bootstrap.cc:654] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:24.363106 15311 log.cc:824] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:24.363188 15313 tablet_bootstrap.cc:492] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219: Bootstrap starting.
I20260924 07:14:24.363760 15313 tablet_bootstrap.cc:654] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:24.364323 15313 log.cc:824] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:24.364499 15310 raft_consensus.cc:348] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53289899844449b2b4e13b9e84a75219" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45667 } } peers { permanent_uuid: "37e5ba8a90b347ce9fcb51fce3fada8e" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 39939 } } peers { permanent_uuid: "5e7c002de03149f6939c8b6df41bbac9" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46809 } }
I20260924 07:14:24.364673 15310 raft_consensus.cc:374] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:24.364719 15310 raft_consensus.cc:729] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e7c002de03149f6939c8b6df41bbac9, State: Initialized, Role: FOLLOWER
I20260924 07:14:24.364830 15310 consensus_queue.cc:260] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53289899844449b2b4e13b9e84a75219" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45667 } } peers { permanent_uuid: "37e5ba8a90b347ce9fcb51fce3fada8e" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 39939 } } peers { permanent_uuid: "5e7c002de03149f6939c8b6df41bbac9" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46809 } }
I20260924 07:14:24.364980 15313 tablet_bootstrap.cc:492] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219: No bootstrap required, opened a new log
I20260924 07:14:24.365043 15313 ts_tablet_manager.cc:1397] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:24.365065 15163 heartbeater.cc:503] Master 127.10.130.254:43513 was elected leader, sending a full tablet report...
I20260924 07:14:24.365063 15310 ts_tablet_manager.cc:1428] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:14:24.365582 15311 tablet_bootstrap.cc:492] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e: No bootstrap required, opened a new log
I20260924 07:14:24.365674 15311 ts_tablet_manager.cc:1397] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260924 07:14:24.367595 15311 raft_consensus.cc:348] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53289899844449b2b4e13b9e84a75219" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45667 } } peers { permanent_uuid: "37e5ba8a90b347ce9fcb51fce3fada8e" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 39939 } } peers { permanent_uuid: "5e7c002de03149f6939c8b6df41bbac9" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46809 } }
I20260924 07:14:24.367794 15311 raft_consensus.cc:374] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:24.367841 15311 raft_consensus.cc:729] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37e5ba8a90b347ce9fcb51fce3fada8e, State: Initialized, Role: FOLLOWER
I20260924 07:14:24.367785 15313 raft_consensus.cc:348] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53289899844449b2b4e13b9e84a75219" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45667 } } peers { permanent_uuid: "37e5ba8a90b347ce9fcb51fce3fada8e" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 39939 } } peers { permanent_uuid: "5e7c002de03149f6939c8b6df41bbac9" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46809 } }
I20260924 07:14:24.367914 15313 raft_consensus.cc:374] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:24.367954 15313 raft_consensus.cc:729] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53289899844449b2b4e13b9e84a75219, State: Initialized, Role: FOLLOWER
I20260924 07:14:24.368033 15311 consensus_queue.cc:260] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53289899844449b2b4e13b9e84a75219" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45667 } } peers { permanent_uuid: "37e5ba8a90b347ce9fcb51fce3fada8e" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 39939 } } peers { permanent_uuid: "5e7c002de03149f6939c8b6df41bbac9" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46809 } }
I20260924 07:14:24.368067 15313 consensus_queue.cc:260] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53289899844449b2b4e13b9e84a75219" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45667 } } peers { permanent_uuid: "37e5ba8a90b347ce9fcb51fce3fada8e" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 39939 } } peers { permanent_uuid: "5e7c002de03149f6939c8b6df41bbac9" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46809 } }
I20260924 07:14:24.368299 15313 ts_tablet_manager.cc:1428] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:14:24.368294 15311 ts_tablet_manager.cc:1428] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260924 07:14:24.368398 15296 heartbeater.cc:503] Master 127.10.130.254:43513 was elected leader, sending a full tablet report...
I20260924 07:14:24.368716 15030 heartbeater.cc:503] Master 127.10.130.254:43513 was elected leader, sending a full tablet report...
W20260924 07:14:24.429025 15164 tablet.cc:2369] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:14:24.496204 15316 raft_consensus.cc:482] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:24.496320 15316 raft_consensus.cc:504] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53289899844449b2b4e13b9e84a75219" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45667 } } peers { permanent_uuid: "37e5ba8a90b347ce9fcb51fce3fada8e" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 39939 } } peers { permanent_uuid: "5e7c002de03149f6939c8b6df41bbac9" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46809 } }
I20260924 07:14:24.496651 15316 leader_election.cc:288] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53289899844449b2b4e13b9e84a75219 (127.10.130.195:45667), 37e5ba8a90b347ce9fcb51fce3fada8e (127.10.130.193:39939)
I20260924 07:14:24.499231 15165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.496812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:45667 (local address 127.10.130.194:33943)
0924 07:14:24.496859 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:24.496871 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:24.496911 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:24.497033 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:24.497035 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:24.497037 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:24.497101 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.497102 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.497688 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.497689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.498482 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.498485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.498996 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.498997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.499075 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.499078 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:24.499122 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 07:14:24.499160 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":5}
I20260924 07:14:24.499230 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.496876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:33943 (local address 127.10.130.195:45667)
0924 07:14:24.496923 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:24.496925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:24.496933 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:24.496956 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:24.496957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:24.496963 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:24.497017 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:24.497158 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.497621 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.498559 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.498927 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.498987 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.498995 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:24.499131 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 07:14:24.499165 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 07:14:24.499378 15319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.496915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:39939 (local address 127.10.130.194:33951)
0924 07:14:24.497063 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:24.497071 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:24.497111 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:24.497244 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:24.497245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:24.497246 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:24.497311 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.497312 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.497926 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.497927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.498653 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:24.498655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.499149 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.499150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:24.499214 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.499217 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:24.499264 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 07:14:24.499299 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260924 07:14:24.499378 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.497049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:33951 (local address 127.10.130.193:39939)
0924 07:14:24.497089 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:24.497091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:24.497101 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:14:24.497184 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:24.497185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:24.497193 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:24.497235 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:24.497363 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.497852 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.498734 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.499077 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.499136 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.499144 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:24.499286 (+ 142us) server_negotiation.cc:299] Negotiation successful
0924 07:14:24.499319 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 07:14:24.499557 15250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74cb56098e77413fa4d99658245afb92" candidate_uuid: "5e7c002de03149f6939c8b6df41bbac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53289899844449b2b4e13b9e84a75219" is_pre_election: true
I20260924 07:14:24.499719 15250 raft_consensus.cc:2454] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e7c002de03149f6939c8b6df41bbac9 in term 0.
I20260924 07:14:24.499939 15051 leader_election.cc:302] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53289899844449b2b4e13b9e84a75219, 5e7c002de03149f6939c8b6df41bbac9; no voters:
I20260924 07:14:24.500080 15316 raft_consensus.cc:2781] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:24.500155 15316 raft_consensus.cc:482] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:24.500173 15316 raft_consensus.cc:3037] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:24.500203 14984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74cb56098e77413fa4d99658245afb92" candidate_uuid: "5e7c002de03149f6939c8b6df41bbac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37e5ba8a90b347ce9fcb51fce3fada8e" is_pre_election: true
I20260924 07:14:24.500339 14984 raft_consensus.cc:2454] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e7c002de03149f6939c8b6df41bbac9 in term 0.
I20260924 07:14:24.500994 15316 raft_consensus.cc:504] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53289899844449b2b4e13b9e84a75219" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45667 } } peers { permanent_uuid: "37e5ba8a90b347ce9fcb51fce3fada8e" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 39939 } } peers { permanent_uuid: "5e7c002de03149f6939c8b6df41bbac9" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46809 } }
I20260924 07:14:24.501092 15316 leader_election.cc:288] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [CANDIDATE]: Term 1 election: Requested vote from peers 53289899844449b2b4e13b9e84a75219 (127.10.130.195:45667), 37e5ba8a90b347ce9fcb51fce3fada8e (127.10.130.193:39939)
I20260924 07:14:24.501256 14984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74cb56098e77413fa4d99658245afb92" candidate_uuid: "5e7c002de03149f6939c8b6df41bbac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37e5ba8a90b347ce9fcb51fce3fada8e"
I20260924 07:14:24.501276 15250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74cb56098e77413fa4d99658245afb92" candidate_uuid: "5e7c002de03149f6939c8b6df41bbac9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53289899844449b2b4e13b9e84a75219"
I20260924 07:14:24.501325 14984 raft_consensus.cc:3037] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:24.501340 15250 raft_consensus.cc:3037] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:24.501997 14984 raft_consensus.cc:2454] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e7c002de03149f6939c8b6df41bbac9 in term 1.
I20260924 07:14:24.502002 15250 raft_consensus.cc:2454] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e7c002de03149f6939c8b6df41bbac9 in term 1.
I20260924 07:14:24.502136 15051 leader_election.cc:302] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37e5ba8a90b347ce9fcb51fce3fada8e, 5e7c002de03149f6939c8b6df41bbac9; no voters:
I20260924 07:14:24.502233 15316 raft_consensus.cc:2781] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:24.502394 15316 raft_consensus.cc:686] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [term 1 LEADER]: Becoming Leader. State: Replica: 5e7c002de03149f6939c8b6df41bbac9, State: Running, Role: LEADER
I20260924 07:14:24.502481 15316 consensus_queue.cc:237] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53289899844449b2b4e13b9e84a75219" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45667 } } peers { permanent_uuid: "37e5ba8a90b347ce9fcb51fce3fada8e" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 39939 } } peers { permanent_uuid: "5e7c002de03149f6939c8b6df41bbac9" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46809 } }
I20260924 07:14:24.503262 14840 catalog_manager.cc:5495] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e7c002de03149f6939c8b6df41bbac9 (127.10.130.194). New cstate: current_term: 1 leader_uuid: "5e7c002de03149f6939c8b6df41bbac9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53289899844449b2b4e13b9e84a75219" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37e5ba8a90b347ce9fcb51fce3fada8e" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 39939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e7c002de03149f6939c8b6df41bbac9" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46809 } health_report { overall_health: HEALTHY } } }
I20260924 07:14:24.535255 15306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.532978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49584 (local address 127.10.130.194:46809)
0924 07:14:24.533029 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:24.533031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:24.533038 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:24.533072 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:24.533073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:24.533090 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:24.533149 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:24.533298 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.533759 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.534540 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:24.534723 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:24.534783 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:24.534921 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:24.535021 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:24.535091 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:24.535137 (+ 46us) server_negotiation.cc:299] Negotiation successful
0924 07:14:24.535156 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 07:14:24.536988 15250 raft_consensus.cc:1261] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219 [term 1 FOLLOWER]: Refusing update from remote peer 5e7c002de03149f6939c8b6df41bbac9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:24.537017 14984 raft_consensus.cc:1261] T 74cb56098e77413fa4d99658245afb92 P 37e5ba8a90b347ce9fcb51fce3fada8e [term 1 FOLLOWER]: Refusing update from remote peer 5e7c002de03149f6939c8b6df41bbac9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:24.537338 15320 consensus_queue.cc:1035] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53289899844449b2b4e13b9e84a75219" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:24.537467 15316 consensus_queue.cc:1035] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37e5ba8a90b347ce9fcb51fce3fada8e" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 39939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:24.542027 14839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:24.542105 14839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:24.542739 14839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aacd0099ec4140ca88275c5031d505d1: Sending DeleteTablet for 3 replicas of tablet 74cb56098e77413fa4d99658245afb92
I20260924 07:14:24.543045 15230 tablet_service.cc:1514] Processing DeleteTablet for tablet 74cb56098e77413fa4d99658245afb92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:24 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:53914
I20260924 07:14:24.543087 15097 tablet_service.cc:1514] Processing DeleteTablet for tablet 74cb56098e77413fa4d99658245afb92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:24 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:49576
I20260924 07:14:24.543080 14964 tablet_service.cc:1514] Processing DeleteTablet for tablet 74cb56098e77413fa4d99658245afb92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:24 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:54212
I20260924 07:14:24.543308 15332 tablet_replica.cc:330] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219: stopping tablet replica
I20260924 07:14:24.543397 15332 raft_consensus.cc:2229] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:24.543493 15332 raft_consensus.cc:2258] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:24.543529 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 14902
I20260924 07:14:24.544371 15332 ts_tablet_manager.cc:1905] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:24.545665 15332 ts_tablet_manager.cc:1918] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:24.545723 15332 log.cc:1196] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/74cb56098e77413fa4d99658245afb92
I20260924 07:14:24.545938 15332 ts_tablet_manager.cc:1939] T 74cb56098e77413fa4d99658245afb92 P 53289899844449b2b4e13b9e84a75219: Deleting consensus metadata
I20260924 07:14:24.546267 14825 catalog_manager.cc:4841] TS 53289899844449b2b4e13b9e84a75219 (127.10.130.195:45667): tablet 74cb56098e77413fa4d99658245afb92 (table test-table [id=c192204c572f4e57be5c4baf23aaf0a2]) successfully deleted
I20260924 07:14:24.546864 15334 tablet_replica.cc:330] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9: stopping tablet replica
I20260924 07:14:24.546960 15334 raft_consensus.cc:2229] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:24.547048 15334 raft_consensus.cc:2258] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:24.547425 15334 ts_tablet_manager.cc:1905] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:24.548915 15334 ts_tablet_manager.cc:1918] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:24.548985 15334 log.cc:1196] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/74cb56098e77413fa4d99658245afb92
I20260924 07:14:24.549256 15334 ts_tablet_manager.cc:1939] T 74cb56098e77413fa4d99658245afb92 P 5e7c002de03149f6939c8b6df41bbac9: Deleting consensus metadata
I20260924 07:14:24.549612 14824 catalog_manager.cc:4841] TS 5e7c002de03149f6939c8b6df41bbac9 (127.10.130.194:46809): tablet 74cb56098e77413fa4d99658245afb92 (table test-table [id=c192204c572f4e57be5c4baf23aaf0a2]) successfully deleted
W20260924 07:14:24.550503 14825 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:39939 (error 108)
I20260924 07:14:24.550661 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 15035
I20260924 07:14:24.550902 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.550806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:39939 (local address 127.0.0.1:54214)
0924 07:14:24.550847 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:24.550873 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:39939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 07:14:24.550979 14825 catalog_manager.cc:4584] TS 37e5ba8a90b347ce9fcb51fce3fada8e (127.10.130.193:39939): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 74cb56098e77413fa4d99658245afb92: Network error: Client connection negotiation failed: client connection to 127.10.130.193:39939: connect: Connection refused (error 111)
I20260924 07:14:24.555254 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 15168
I20260924 07:14:24.558848 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 14809
2026-09-24T07:14:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1877 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-24T07:14:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:24Z Disabled control of system clock
I20260924 07:14:24.582839 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:35195
--webserver_interface=127.10.130.254
--webserver_port=0
--builtin_ntp_servers=127.10.130.212:38969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:35195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 07:14:24.659734 15339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:24.659929 15339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:24.659948 15339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:24.661074 15339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:24.661114 15339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:24.661125 15339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:24.661136 15339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:24.661146 15339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:24.662405 15339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:35195
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:35195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:24.662587 15339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:24.662806 15339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:24.667205 15347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:24.667218 15344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:24.667258 15345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:24.667366 15339 server_base.cc:1034] running on GCE node
I20260924 07:14:24.667682 15339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:24.667953 15339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:24.669122 15339 hybrid_clock.cc:648] HybridClock initialized: now 1790234064669109 us; error 29 us; skew 500 ppm
I20260924 07:14:24.670291 15339 webserver.cc:460] Webserver started at http://127.10.130.254:45781/ using document root <none> and password file <none>
I20260924 07:14:24.670493 15339 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:24.670542 15339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:24.670667 15339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:24.671696 15339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "2cc49f6ed5c14aea82a2791ab198104a"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "dc85d3f3998794f21e9e9b218d42e481"
server_key_iv: "a2633a91e8c67a98ffe13e6a1d3381eb"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.672045 15339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "2cc49f6ed5c14aea82a2791ab198104a"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "dc85d3f3998794f21e9e9b218d42e481"
server_key_iv: "a2633a91e8c67a98ffe13e6a1d3381eb"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.673233 15339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 07:14:24.673923 15353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.674069 15339 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:24.674151 15339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "2cc49f6ed5c14aea82a2791ab198104a"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "dc85d3f3998794f21e9e9b218d42e481"
server_key_iv: "a2633a91e8c67a98ffe13e6a1d3381eb"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.674213 15339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:24.677625 15339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:24.677870 15339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:24.677992 15339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:24.681402 15339 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:35195
I20260924 07:14:24.681432 15405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:35195 every 8 connection(s)
I20260924 07:14:24.681720 15339 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:24.682309 15406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:24.684967 15406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a: Bootstrap starting.
I20260924 07:14:24.685496 15406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:24.685760 15406 log.cc:824] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:24.686381 15406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a: No bootstrap required, opened a new log
I20260924 07:14:24.686379 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 15339
I20260924 07:14:24.686463 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:24.686638 10763 external_mini_cluster.cc:1428] Setting key f6aff9d9b3adbed834b4b10ba768ceab
I20260924 07:14:24.688323 15406 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc49f6ed5c14aea82a2791ab198104a" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 35195 } }
I20260924 07:14:24.688493 15406 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:24.688527 15406 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cc49f6ed5c14aea82a2791ab198104a, State: Initialized, Role: FOLLOWER
I20260924 07:14:24.688683 15406 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc49f6ed5c14aea82a2791ab198104a" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 35195 } }
I20260924 07:14:24.688776 15406 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:24.688834 15406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:24.688874 15406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:24.689488 15406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc49f6ed5c14aea82a2791ab198104a" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 35195 } }
I20260924 07:14:24.689595 15406 leader_election.cc:302] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cc49f6ed5c14aea82a2791ab198104a; no voters:
I20260924 07:14:24.689646 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.687131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.10.130.254:35195)
0924 07:14:24.687356 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:24.687365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:24.687398 (+ 33us) server_negotiation.cc:407] Connection header received
0924 07:14:24.688287 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:24.688296 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:24.688347 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:24.688496 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:24.689179 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:24.689182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:24.689183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:24.689187 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:24.689231 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:24.689257 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:24.689330 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:24.689369 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 07:14:24.689424 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":80,"threads_started":1}
I20260924 07:14:24.689735 15406 leader_election.cc:288] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:24.689977 15406 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:24.690143 15412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:24.690276 15412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [term 1 LEADER]: Becoming Leader. State: Replica: 2cc49f6ed5c14aea82a2791ab198104a, State: Running, Role: LEADER
I20260924 07:14:24.690415 15412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc49f6ed5c14aea82a2791ab198104a" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 35195 } }
I20260924 07:14:24.690727 15412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cc49f6ed5c14aea82a2791ab198104a. Latest consensus state: current_term: 1 leader_uuid: "2cc49f6ed5c14aea82a2791ab198104a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc49f6ed5c14aea82a2791ab198104a" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 35195 } } }
I20260924 07:14:24.690755 15413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cc49f6ed5c14aea82a2791ab198104a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cc49f6ed5c14aea82a2791ab198104a" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 35195 } } }
I20260924 07:14:24.690869 15412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:24.690956 15413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a [sys.catalog]: This master's current role is: LEADER
W20260924 07:14:24.691713 15426 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:14:24.691756 15426 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:14:24.691814 15424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:24.692042 15424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:24.693455 15424 catalog_manager.cc:1347] Generated new cluster ID: 333ea90c95314044842dfd0f3416daff
I20260924 07:14:24.693506 15424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:24.698402 15424 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:24.698547 15424 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:24.702623 15424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a: Generated new TSK 0
I20260924 07:14:24.702781 15424 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:14:24.708182 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:35195
--builtin_ntp_servers=127.10.130.212:38969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 07:14:24.784085 15430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:24.784271 15430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:24.784291 15430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:24.785430 15430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:24.785466 15430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:24.785490 15430 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:24.786767 15430 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:24.786964 15430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:24.787231 15430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:24.791929 15435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:24.792016 15436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:24.792084 15438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:24.792346 15430 server_base.cc:1034] running on GCE node
I20260924 07:14:24.792536 15430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:24.792747 15430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:24.793884 15430 hybrid_clock.cc:648] HybridClock initialized: now 1790234064793871 us; error 31 us; skew 500 ppm
I20260924 07:14:24.795010 15430 webserver.cc:460] Webserver started at http://127.10.130.193:40569/ using document root <none> and password file <none>
I20260924 07:14:24.795221 15430 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:24.795269 15430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:24.795390 15430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:24.796445 15430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "88c9d673920d4df6baef0bb2b6ccff25"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "113e5eed75862cd8deb8347b1ab341a3"
server_key_iv: "dc3d6d29eb51506d3d0f310d5213314a"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.796756 15430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "88c9d673920d4df6baef0bb2b6ccff25"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "113e5eed75862cd8deb8347b1ab341a3"
server_key_iv: "dc3d6d29eb51506d3d0f310d5213314a"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.797937 15430 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:24.798614 15444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.798774 15430 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:24.798861 15430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "88c9d673920d4df6baef0bb2b6ccff25"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "113e5eed75862cd8deb8347b1ab341a3"
server_key_iv: "dc3d6d29eb51506d3d0f310d5213314a"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.798923 15430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:24.801991 15430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:24.802256 15430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:24.802368 15430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:24.802551 15430 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:24.802809 15430 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:24.802839 15430 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.802865 15430 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:24.802888 15430 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.808145 15430 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:42275
I20260924 07:14:24.808200 15557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:42275 every 8 connection(s)
I20260924 07:14:24.808488 15430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:14:24.811290 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.809167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:53685 (local address 127.10.130.254:35195)
0924 07:14:24.809206 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:24.809208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:24.809467 (+ 259us) server_negotiation.cc:407] Connection header received
0924 07:14:24.809958 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:24.809959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:24.809966 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:24.810034 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:24.811084 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:24.811085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:24.811085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:24.811090 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:24.811105 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:24.811109 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:24.811162 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:24.811204 (+ 42us) server_negotiation.cc:299] Negotiation successful
0924 07:14:24.811217 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:14:24.811432 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.809182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:35195 (local address 127.10.130.193:53685)
0924 07:14:24.809364 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:24.809395 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:24.809863 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:24.810088 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:24.810091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:24.810099 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:24.810946 (+ 847us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:24.810947 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:24.810975 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:24.811004 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:24.811158 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:24.811161 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:24.811163 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:14:24.811209 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 07:14:24.811255 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260924 07:14:24.811862 15558 heartbeater.cc:347] Connected to a master server at 127.10.130.254:35195
I20260924 07:14:24.811955 15558 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:24.812124 15558 heartbeater.cc:511] Master 127.10.130.254:35195 requested a full tablet report, sending...
I20260924 07:14:24.812280 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 15430
I20260924 07:14:24.812372 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:24.812556 10763 external_mini_cluster.cc:1428] Setting key 3b1474c75fac06f2f4921e5130996b89
I20260924 07:14:24.812677 15370 ts_manager.cc:194] Registered new tserver with Master: 88c9d673920d4df6baef0bb2b6ccff25 (127.10.130.193:42275)
I20260924 07:14:24.813488 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:35195
--builtin_ntp_servers=127.10.130.212:38969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 07:14:24.889688 15561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:24.889886 15561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:24.889914 15561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:24.891103 15561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:24.891140 15561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:24.891175 15561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:24.892515 15561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:24.892743 15561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:24.893004 15561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:24.897569 15567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:24.897573 15566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:24.897593 15569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:24.897797 15561 server_base.cc:1034] running on GCE node
I20260924 07:14:24.897989 15561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:24.898196 15561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:24.899351 15561 hybrid_clock.cc:648] HybridClock initialized: now 1790234064899335 us; error 29 us; skew 500 ppm
I20260924 07:14:24.900599 15561 webserver.cc:460] Webserver started at http://127.10.130.194:41425/ using document root <none> and password file <none>
I20260924 07:14:24.900815 15561 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:24.900861 15561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:24.900979 15561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:24.902020 15561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "c7f80598102545aa899eadf7376582e3"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "4a5afef591e4c3a66adbbb7aeaa9f2a2"
server_key_iv: "76b5124715bdf562846f8bb7a760c61c"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.902355 15561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "c7f80598102545aa899eadf7376582e3"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "4a5afef591e4c3a66adbbb7aeaa9f2a2"
server_key_iv: "76b5124715bdf562846f8bb7a760c61c"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.903475 15561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:14:24.904103 15575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.904241 15561 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.904316 15561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "c7f80598102545aa899eadf7376582e3"
format_stamp: "Formatted at 2026-09-24 07:14:24 on dist-test-slave-7q7m"
server_key: "4a5afef591e4c3a66adbbb7aeaa9f2a2"
server_key_iv: "76b5124715bdf562846f8bb7a760c61c"
server_key_version: "encryptionkey@0"
I20260924 07:14:24.904382 15561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:24.907497 15561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:24.907727 15561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:24.907837 15561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:24.908036 15561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:24.908308 15561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:24.908337 15561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.908365 15561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:24.908387 15561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:24.913589 15561 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:37017
I20260924 07:14:24.913631 15688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:37017 every 8 connection(s)
I20260924 07:14:24.913918 15561 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:14:24.916667 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.914531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:53047 (local address 127.10.130.254:35195)
0924 07:14:24.914649 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:24.914651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:24.914825 (+ 174us) server_negotiation.cc:407] Connection header received
0924 07:14:24.915289 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:24.915290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:24.915297 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:24.915346 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:24.916436 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:24.916437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:24.916438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:24.916442 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:24.916458 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:24.916462 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:24.916511 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:24.916575 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 07:14:24.916596 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260924 07:14:24.916838 15691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:24.914510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:35195 (local address 127.10.130.194:53047)
0924 07:14:24.914701 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:24.914739 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:24.915197 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:24.915414 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:24.915417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:24.915427 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:24.916301 (+ 874us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:24.916302 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:24.916329 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:24.916355 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:24.916512 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:24.916517 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:24.916520 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:24.916586 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 07:14:24.916644 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260924 07:14:24.917268 15689 heartbeater.cc:347] Connected to a master server at 127.10.130.254:35195
I20260924 07:14:24.917349 15689 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:24.917379 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 15561
I20260924 07:14:24.917456 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:24.917492 15689 heartbeater.cc:511] Master 127.10.130.254:35195 requested a full tablet report, sending...
I20260924 07:14:24.917589 10763 external_mini_cluster.cc:1428] Setting key 6070d4dfbbcee98c40f19150c083d888
I20260924 07:14:24.917857 15369 ts_manager.cc:194] Registered new tserver with Master: c7f80598102545aa899eadf7376582e3 (127.10.130.194:37017)
I20260924 07:14:24.918409 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:35195
--builtin_ntp_servers=127.10.130.212:38969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 07:14:24.995209 15692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:24.995388 15692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:24.995406 15692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:24.996534 15692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:24.996569 15692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:24.996594 15692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:24.997824 15692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:35195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:24.997995 15692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:24.998227 15692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:25.002791 15697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:25.002844 15698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:25.002884 15692 server_base.cc:1034] running on GCE node
W20260924 07:14:25.002791 15700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:25.003158 15692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:25.003384 15692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:25.004510 15692 hybrid_clock.cc:648] HybridClock initialized: now 1790234065004500 us; error 29 us; skew 500 ppm
I20260924 07:14:25.005645 15692 webserver.cc:460] Webserver started at http://127.10.130.195:37961/ using document root <none> and password file <none>
I20260924 07:14:25.005854 15692 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:25.005900 15692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:25.006021 15692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:25.007011 15692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "faeb4f3af3de475ba3ef052796e5432e"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "37006f3f42dd81fa9cb80a717301c565"
server_key_iv: "f02b9c82aed4bfcbca9f4510d81c7f06"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.007341 15692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "faeb4f3af3de475ba3ef052796e5432e"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "37006f3f42dd81fa9cb80a717301c565"
server_key_iv: "f02b9c82aed4bfcbca9f4510d81c7f06"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.008563 15692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:14:25.009186 15706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:25.009322 15692 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:25.009399 15692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "faeb4f3af3de475ba3ef052796e5432e"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "37006f3f42dd81fa9cb80a717301c565"
server_key_iv: "f02b9c82aed4bfcbca9f4510d81c7f06"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.009455 15692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:25.012573 15692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:25.012795 15692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:25.012905 15692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:25.013077 15692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:25.013340 15692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:25.013370 15692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:25.013396 15692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:25.013418 15692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:25.018635 15692 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:41653
I20260924 07:14:25.018669 15819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:41653 every 8 connection(s)
I20260924 07:14:25.018954 15692 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:25.021744 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.019584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:33433 (local address 127.10.130.254:35195)
0924 07:14:25.019642 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.019644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.019859 (+ 215us) server_negotiation.cc:407] Connection header received
0924 07:14:25.020363 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.020365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.020371 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.020427 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:25.021510 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:25.021512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:25.021512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:25.021516 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:25.021531 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:25.021535 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:25.021593 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:25.021659 (+ 66us) server_negotiation.cc:299] Negotiation successful
0924 07:14:25.021674 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 07:14:25.021905 15822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.019577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:35195 (local address 127.10.130.195:33433)
0924 07:14:25.019751 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:25.019781 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:25.020270 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:25.020494 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:25.020498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:25.020508 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:25.021372 (+ 864us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:25.021372 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:25.021399 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:25.021425 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:25.021588 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:25.021592 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:25.021595 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:25.021663 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 07:14:25.021722 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260924 07:14:25.022325 15820 heartbeater.cc:347] Connected to a master server at 127.10.130.254:35195
I20260924 07:14:25.022367 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 15692
I20260924 07:14:25.022408 15820 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:25.022449 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:25.022547 15820 heartbeater.cc:511] Master 127.10.130.254:35195 requested a full tablet report, sending...
I20260924 07:14:25.022596 10763 external_mini_cluster.cc:1428] Setting key 1d2a451568f7abd0b692205b592bef4f
I20260924 07:14:25.022876 15370 ts_manager.cc:194] Registered new tserver with Master: faeb4f3af3de475ba3ef052796e5432e (127.10.130.195:41653)
I20260924 07:14:25.023360 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:14:25.025425 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.024571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50278 (local address 127.10.130.254:35195)
0924 07:14:25.024655 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.024657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.024694 (+ 37us) server_negotiation.cc:407] Connection header received
0924 07:14:25.024743 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.024744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.024753 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.024795 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:25.025250 (+ 455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:25.025251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:25.025251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:25.025254 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:25.025262 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:25.025265 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:25.025297 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:25.025355 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 07:14:25.025364 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260924 07:14:25.026772 15370 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50278:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:14:25.028204 15370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:25.031387 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.029991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:37017 (local address 127.0.0.1:45884)
0924 07:14:25.030098 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:25.030135 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:25.030188 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:25.030533 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:25.030538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:25.030544 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:25.030945 (+ 401us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:25.030946 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:25.030961 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:25.030982 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:25.031235 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:25.031240 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:25.031243 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:25.031293 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 07:14:25.031305 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":47,"threads_started":1}
I20260924 07:14:25.031400 15830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.030076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56922 (local address 127.10.130.193:42275)
0924 07:14:25.030603 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.030606 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.030615 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:25.030638 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.030640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.030648 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.030891 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:25.031163 (+ 272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:25.031166 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:25.031167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:25.031170 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:25.031196 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:25.031209 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:25.031267 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:25.031312 (+ 45us) server_negotiation.cc:299] Negotiation successful
0924 07:14:25.031325 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":71,"threads_started":1}
I20260924 07:14:25.031394 15829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.029998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:42275 (local address 127.0.0.1:56922)
0924 07:14:25.030140 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:25.030148 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:25.030188 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:25.030735 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:25.030736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:25.030739 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:25.031083 (+ 344us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:25.031084 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:25.031090 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:25.031103 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:25.031261 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:25.031262 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:25.031263 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 07:14:25.031307 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 07:14:25.031319 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260924 07:14:25.031672 15832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.030240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45884 (local address 127.10.130.194:37017)
0924 07:14:25.030404 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.030408 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.030419 (+ 11us) server_negotiation.cc:407] Connection header received
0924 07:14:25.030444 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.030446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.030454 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.030524 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:25.031074 (+ 550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:25.031082 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:25.031082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:25.031086 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:25.031122 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:25.031147 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:25.031187 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:25.031559 (+ 372us) server_negotiation.cc:299] Negotiation successful
0924 07:14:25.031590 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260924 07:14:25.032131 15492 tablet_service.cc:1467] Processing CreateTablet for tablet ead8381f87bc4eb58143aac7dafaba88 (DEFAULT_TABLE table=test-table [id=2f63c813141045fb818a77d4dd1f610a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:25.032404 15492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ead8381f87bc4eb58143aac7dafaba88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:25.032464 15833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.030305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58668 (local address 127.10.130.195:41653)
0924 07:14:25.030433 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.030437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.030476 (+ 39us) server_negotiation.cc:407] Connection header received
0924 07:14:25.030529 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.030531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.030538 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.030605 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:25.032171 (+ 1566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:25.032177 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:25.032178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:25.032181 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:25.032211 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:25.032229 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:25.032277 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:25.032382 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 07:14:25.032395 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260924 07:14:25.032622 15831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.030159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:41653 (local address 127.0.0.1:58668)
0924 07:14:25.030420 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:25.030429 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:25.030484 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:25.031091 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:25.031092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:25.031094 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:25.032087 (+ 993us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:25.032088 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:25.032094 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:25.032105 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:25.032324 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:25.032325 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:25.032325 (+ 0us) client_negotiation.cc:769] Sending connection context
0924 07:14:25.032371 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 07:14:25.032383 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":58,"threads_started":1}
I20260924 07:14:25.033212 15754 tablet_service.cc:1467] Processing CreateTablet for tablet ead8381f87bc4eb58143aac7dafaba88 (DEFAULT_TABLE table=test-table [id=2f63c813141045fb818a77d4dd1f610a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:25.033473 15754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ead8381f87bc4eb58143aac7dafaba88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:25.034874 15834 tablet_bootstrap.cc:492] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25: Bootstrap starting.
I20260924 07:14:25.035167 15623 tablet_service.cc:1467] Processing CreateTablet for tablet ead8381f87bc4eb58143aac7dafaba88 (DEFAULT_TABLE table=test-table [id=2f63c813141045fb818a77d4dd1f610a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:25.035445 15623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ead8381f87bc4eb58143aac7dafaba88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:25.035691 15834 tablet_bootstrap.cc:654] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:25.035956 15835 tablet_bootstrap.cc:492] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e: Bootstrap starting.
I20260924 07:14:25.036031 15834 log.cc:824] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:25.036676 15835 tablet_bootstrap.cc:654] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:25.036727 15834 tablet_bootstrap.cc:492] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25: No bootstrap required, opened a new log
I20260924 07:14:25.036782 15834 ts_tablet_manager.cc:1397] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:25.036995 15835 log.cc:824] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:25.037711 15835 tablet_bootstrap.cc:492] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e: No bootstrap required, opened a new log
I20260924 07:14:25.037786 15835 ts_tablet_manager.cc:1397] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:14:25.038431 15838 tablet_bootstrap.cc:492] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3: Bootstrap starting.
I20260924 07:14:25.038904 15834 raft_consensus.cc:348] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f80598102545aa899eadf7376582e3" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37017 } } peers { permanent_uuid: "88c9d673920d4df6baef0bb2b6ccff25" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42275 } } peers { permanent_uuid: "faeb4f3af3de475ba3ef052796e5432e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 41653 } }
I20260924 07:14:25.039065 15834 raft_consensus.cc:374] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:25.039091 15838 tablet_bootstrap.cc:654] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:25.039110 15834 raft_consensus.cc:729] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88c9d673920d4df6baef0bb2b6ccff25, State: Initialized, Role: FOLLOWER
I20260924 07:14:25.039243 15834 consensus_queue.cc:260] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f80598102545aa899eadf7376582e3" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37017 } } peers { permanent_uuid: "88c9d673920d4df6baef0bb2b6ccff25" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42275 } } peers { permanent_uuid: "faeb4f3af3de475ba3ef052796e5432e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 41653 } }
I20260924 07:14:25.039376 15838 log.cc:824] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:25.039489 15834 ts_tablet_manager.cc:1428] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:14:25.039642 15558 heartbeater.cc:503] Master 127.10.130.254:35195 was elected leader, sending a full tablet report...
I20260924 07:14:25.040081 15835 raft_consensus.cc:348] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f80598102545aa899eadf7376582e3" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37017 } } peers { permanent_uuid: "88c9d673920d4df6baef0bb2b6ccff25" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42275 } } peers { permanent_uuid: "faeb4f3af3de475ba3ef052796e5432e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 41653 } }
I20260924 07:14:25.040200 15835 raft_consensus.cc:374] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:25.040220 15835 raft_consensus.cc:729] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faeb4f3af3de475ba3ef052796e5432e, State: Initialized, Role: FOLLOWER
I20260924 07:14:25.040323 15835 consensus_queue.cc:260] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f80598102545aa899eadf7376582e3" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37017 } } peers { permanent_uuid: "88c9d673920d4df6baef0bb2b6ccff25" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42275 } } peers { permanent_uuid: "faeb4f3af3de475ba3ef052796e5432e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 41653 } }
I20260924 07:14:25.040575 15835 ts_tablet_manager.cc:1428] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260924 07:14:25.040606 15820 heartbeater.cc:503] Master 127.10.130.254:35195 was elected leader, sending a full tablet report...
I20260924 07:14:25.041131 15838 tablet_bootstrap.cc:492] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3: No bootstrap required, opened a new log
I20260924 07:14:25.041189 15838 ts_tablet_manager.cc:1397] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 07:14:25.043035 15838 raft_consensus.cc:348] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f80598102545aa899eadf7376582e3" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37017 } } peers { permanent_uuid: "88c9d673920d4df6baef0bb2b6ccff25" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42275 } } peers { permanent_uuid: "faeb4f3af3de475ba3ef052796e5432e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 41653 } }
I20260924 07:14:25.043146 15838 raft_consensus.cc:374] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:25.043165 15838 raft_consensus.cc:729] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f80598102545aa899eadf7376582e3, State: Initialized, Role: FOLLOWER
I20260924 07:14:25.043239 15838 consensus_queue.cc:260] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f80598102545aa899eadf7376582e3" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37017 } } peers { permanent_uuid: "88c9d673920d4df6baef0bb2b6ccff25" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42275 } } peers { permanent_uuid: "faeb4f3af3de475ba3ef052796e5432e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 41653 } }
I20260924 07:14:25.043443 15838 ts_tablet_manager.cc:1428] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 07:14:25.043498 15689 heartbeater.cc:503] Master 127.10.130.254:35195 was elected leader, sending a full tablet report...
I20260924 07:14:25.059219 15839 raft_consensus.cc:482] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:25.059326 15839 raft_consensus.cc:504] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f80598102545aa899eadf7376582e3" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37017 } } peers { permanent_uuid: "88c9d673920d4df6baef0bb2b6ccff25" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42275 } } peers { permanent_uuid: "faeb4f3af3de475ba3ef052796e5432e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 41653 } }
W20260924 07:14:25.059540 15559 tablet.cc:2369] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:14:25.059636 15839 leader_election.cc:288] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7f80598102545aa899eadf7376582e3 (127.10.130.194:37017), faeb4f3af3de475ba3ef052796e5432e (127.10.130.195:41653)
I20260924 07:14:25.060837 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.059837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:37017 (local address 127.10.130.193:46113)
0924 07:14:25.059884 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:25.059895 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:25.059931 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:25.060087 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:25.060088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:25.060091 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:25.060528 (+ 437us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:25.060528 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:25.060535 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:25.060545 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:25.060718 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:25.060719 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:25.060720 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 07:14:25.060760 (+ 40us) client_negotiation.cc:240] Negotiation successful
0924 07:14:25.060772 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260924 07:14:25.060837 15832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.059895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:46113 (local address 127.10.130.194:37017)
0924 07:14:25.059940 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.059942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.059953 (+ 11us) server_negotiation.cc:407] Connection header received
0924 07:14:25.060005 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.060006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.060012 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.060067 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:25.060629 (+ 562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:25.060630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:25.060630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:25.060635 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:25.060649 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:25.060653 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:25.060716 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:25.060759 (+ 43us) server_negotiation.cc:299] Negotiation successful
0924 07:14:25.060771 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:14:25.060957 15833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.060059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:49417 (local address 127.10.130.195:41653)
0924 07:14:25.060114 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.060116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.060124 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:25.060185 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.060186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.060190 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.060236 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:25.060763 (+ 527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:25.060764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:25.060765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:25.060770 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:25.060784 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:25.060789 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:25.060836 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:25.060888 (+ 52us) server_negotiation.cc:299] Negotiation successful
0924 07:14:25.060899 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 07:14:25.060969 15843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.059924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:41653 (local address 127.10.130.193:49417)
0924 07:14:25.060081 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:25.060091 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:25.060129 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:25.060247 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:25.060255 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:25.060258 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:25.060658 (+ 400us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:25.060658 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:25.060665 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:25.060678 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:25.060838 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:25.060839 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:25.060840 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 07:14:25.060878 (+ 38us) client_negotiation.cc:240] Negotiation successful
0924 07:14:25.060891 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":113,"threads_started":1}
I20260924 07:14:25.061110 15643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ead8381f87bc4eb58143aac7dafaba88" candidate_uuid: "88c9d673920d4df6baef0bb2b6ccff25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f80598102545aa899eadf7376582e3" is_pre_election: true
I20260924 07:14:25.061239 15643 raft_consensus.cc:2454] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88c9d673920d4df6baef0bb2b6ccff25 in term 0.
I20260924 07:14:25.061448 15445 leader_election.cc:302] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88c9d673920d4df6baef0bb2b6ccff25, c7f80598102545aa899eadf7376582e3; no voters:
I20260924 07:14:25.061559 15839 raft_consensus.cc:2781] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:25.061551 15774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ead8381f87bc4eb58143aac7dafaba88" candidate_uuid: "88c9d673920d4df6baef0bb2b6ccff25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faeb4f3af3de475ba3ef052796e5432e" is_pre_election: true
I20260924 07:14:25.061614 15839 raft_consensus.cc:482] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:25.061630 15839 raft_consensus.cc:3037] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:25.061671 15774 raft_consensus.cc:2454] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88c9d673920d4df6baef0bb2b6ccff25 in term 0.
I20260924 07:14:25.062319 15839 raft_consensus.cc:504] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f80598102545aa899eadf7376582e3" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37017 } } peers { permanent_uuid: "88c9d673920d4df6baef0bb2b6ccff25" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42275 } } peers { permanent_uuid: "faeb4f3af3de475ba3ef052796e5432e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 41653 } }
I20260924 07:14:25.062419 15839 leader_election.cc:288] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [CANDIDATE]: Term 1 election: Requested vote from peers c7f80598102545aa899eadf7376582e3 (127.10.130.194:37017), faeb4f3af3de475ba3ef052796e5432e (127.10.130.195:41653)
I20260924 07:14:25.062556 15643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ead8381f87bc4eb58143aac7dafaba88" candidate_uuid: "88c9d673920d4df6baef0bb2b6ccff25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7f80598102545aa899eadf7376582e3"
I20260924 07:14:25.062564 15774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ead8381f87bc4eb58143aac7dafaba88" candidate_uuid: "88c9d673920d4df6baef0bb2b6ccff25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "faeb4f3af3de475ba3ef052796e5432e"
I20260924 07:14:25.062623 15643 raft_consensus.cc:3037] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:25.062628 15774 raft_consensus.cc:3037] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:25.063272 15643 raft_consensus.cc:2454] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88c9d673920d4df6baef0bb2b6ccff25 in term 1.
I20260924 07:14:25.063282 15774 raft_consensus.cc:2454] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88c9d673920d4df6baef0bb2b6ccff25 in term 1.
I20260924 07:14:25.063409 15445 leader_election.cc:302] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88c9d673920d4df6baef0bb2b6ccff25, c7f80598102545aa899eadf7376582e3; no voters:
I20260924 07:14:25.063503 15839 raft_consensus.cc:2781] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:25.063640 15839 raft_consensus.cc:686] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [term 1 LEADER]: Becoming Leader. State: Replica: 88c9d673920d4df6baef0bb2b6ccff25, State: Running, Role: LEADER
I20260924 07:14:25.063738 15839 consensus_queue.cc:237] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f80598102545aa899eadf7376582e3" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37017 } } peers { permanent_uuid: "88c9d673920d4df6baef0bb2b6ccff25" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42275 } } peers { permanent_uuid: "faeb4f3af3de475ba3ef052796e5432e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 41653 } }
I20260924 07:14:25.064477 15370 catalog_manager.cc:5495] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 88c9d673920d4df6baef0bb2b6ccff25 (127.10.130.193). New cstate: current_term: 1 leader_uuid: "88c9d673920d4df6baef0bb2b6ccff25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7f80598102545aa899eadf7376582e3" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88c9d673920d4df6baef0bb2b6ccff25" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "faeb4f3af3de475ba3ef052796e5432e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 41653 } health_report { overall_health: UNKNOWN } } }
I20260924 07:14:25.067538 15830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.066875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56930 (local address 127.10.130.193:42275)
0924 07:14:25.066916 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.066917 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.066924 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:25.066951 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.066952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.066962 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.067005 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:25.067401 (+ 396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:25.067402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:25.067402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:25.067406 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:25.067417 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:25.067420 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:25.067460 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:25.067487 (+ 27us) server_negotiation.cc:299] Negotiation successful
0924 07:14:25.067496 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 07:14:25.068900 15774 raft_consensus.cc:1261] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e [term 1 FOLLOWER]: Refusing update from remote peer 88c9d673920d4df6baef0bb2b6ccff25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:25.068907 15643 raft_consensus.cc:1261] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3 [term 1 FOLLOWER]: Refusing update from remote peer 88c9d673920d4df6baef0bb2b6ccff25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:25.069170 15844 consensus_queue.cc:1035] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7f80598102545aa899eadf7376582e3" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:25.069285 15839 consensus_queue.cc:1035] T ead8381f87bc4eb58143aac7dafaba88 P 88c9d673920d4df6baef0bb2b6ccff25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "faeb4f3af3de475ba3ef052796e5432e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 41653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:25.071569 15847 mvcc.cc:205] Tried to move back new op lower bound from 7332798730520203264 to 7332798730501533696. Current Snapshot: MvccSnapshot[applied={T|T < 7332798730520203264}]
I20260924 07:14:25.071868 15848 mvcc.cc:205] Tried to move back new op lower bound from 7332798730520203264 to 7332798730501533696. Current Snapshot: MvccSnapshot[applied={T|T < 7332798730520203264}]
I20260924 07:14:25.072151 15849 mvcc.cc:205] Tried to move back new op lower bound from 7332798730520203264 to 7332798730501533696. Current Snapshot: MvccSnapshot[applied={T|T < 7332798730520203264}]
I20260924 07:14:25.073891 15370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:25.073969 15370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50278:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:25.074585 15370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2cc49f6ed5c14aea82a2791ab198104a: Sending DeleteTablet for 3 replicas of tablet ead8381f87bc4eb58143aac7dafaba88
I20260924 07:14:25.074915 15623 tablet_service.cc:1514] Processing DeleteTablet for tablet ead8381f87bc4eb58143aac7dafaba88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:25 UTC) from {username='slave'} at 127.0.0.1:45884
I20260924 07:14:25.074927 15492 tablet_service.cc:1514] Processing DeleteTablet for tablet ead8381f87bc4eb58143aac7dafaba88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:25 UTC) from {username='slave'} at 127.0.0.1:56922
I20260924 07:14:25.075111 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 15430
I20260924 07:14:25.075139 15857 tablet_replica.cc:330] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3: stopping tablet replica
I20260924 07:14:25.075132 15754 tablet_service.cc:1514] Processing DeleteTablet for tablet ead8381f87bc4eb58143aac7dafaba88 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:25 UTC) from {username='slave'} at 127.0.0.1:58668
W20260924 07:14:25.080271 15355 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:42275 (error 108)
I20260924 07:14:25.080542 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 15561
I20260924 07:14:25.080552 15857 raft_consensus.cc:2229] T ead8381f87bc4eb58143aac7dafaba88 P c7f80598102545aa899eadf7376582e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:25.081074 15831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.080604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:42275 (local address 127.0.0.1:56932)
0924 07:14:25.081004 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:25.081042 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:42275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383}
I20260924 07:14:25.081248 15859 tablet_replica.cc:330] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e: stopping tablet replica
W20260924 07:14:25.081243 15355 catalog_manager.cc:4584] TS 88c9d673920d4df6baef0bb2b6ccff25 (127.10.130.193:42275): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ead8381f87bc4eb58143aac7dafaba88: Network error: Client connection negotiation failed: client connection to 127.10.130.193:42275: connect: Connection refused (error 111)
I20260924 07:14:25.081331 15859 raft_consensus.cc:2229] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:25.081415 15859 raft_consensus.cc:2258] T ead8381f87bc4eb58143aac7dafaba88 P faeb4f3af3de475ba3ef052796e5432e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:25.084523 15831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.084433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:37017 (local address 127.0.0.1:45900)
0924 07:14:25.084473 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:25.084495 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.194:37017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260924 07:14:25.084802 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 15692
I20260924 07:14:25.088346 15831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.088244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:41653 (local address 127.0.0.1:58680)
0924 07:14:25.088297 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:25.088321 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.195:41653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 07:14:25.088340 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 15339
2026-09-24T07:14:25Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (529 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260924 07:14:25.106314 10763 mini_oidc.cc:152] Starting JWKS server
I20260924 07:14:25.107333 10763 webserver.cc:460] Webserver started at https://127.0.0.1:37585/ using document root <none> and password file <none>
I20260924 07:14:25.107411 10763 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 07:14:25.107558 10763 webserver.cc:460] Webserver started at http://127.0.0.1:36169/ using document root <none> and password file <none>
2026-09-24T07:14:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:25Z Disabled control of system clock
I20260924 07:14:25.114482 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:37905
--webserver_interface=127.10.130.254
--webserver_port=0
--builtin_ntp_servers=127.10.130.212:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:37905
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37585/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/testchainca.cert with env {}
W20260924 07:14:25.190797 15866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:25.190999 15866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:25.191027 15866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:25.192198 15866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:25.192236 15866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:25.192252 15866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:25.192268 15866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:25.192282 15866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 07:14:25.192298 15866 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 07:14:25.192320 15866 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37585/jwks.json
I20260924 07:14:25.193835 15866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:37905
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:37905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37585/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:25.194073 15866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:25.194259 15866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:25.198662 15874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:25.198748 15872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:25.198654 15871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:25.198922 15866 server_base.cc:1034] running on GCE node
I20260924 07:14:25.199132 15866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:25.199363 15866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:25.200490 15866 hybrid_clock.cc:648] HybridClock initialized: now 1790234065200478 us; error 30 us; skew 500 ppm
I20260924 07:14:25.201565 15866 webserver.cc:460] Webserver started at http://127.10.130.254:34817/ using document root <none> and password file <none>
I20260924 07:14:25.201738 15866 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:25.201784 15866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:25.201870 15866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:25.202816 15866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "a6ffb7e2a4ef4486af62f99a1c8b72db"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "ba441ab3190f6009a910b708d4a1c289"
server_key_iv: "d28ab4e7d0400d4f450f5cd7d490e20a"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.203099 15866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "a6ffb7e2a4ef4486af62f99a1c8b72db"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "ba441ab3190f6009a910b708d4a1c289"
server_key_iv: "d28ab4e7d0400d4f450f5cd7d490e20a"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.204259 15866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:25.204901 15880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:25.205044 15866 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:25.205120 15866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "a6ffb7e2a4ef4486af62f99a1c8b72db"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "ba441ab3190f6009a910b708d4a1c289"
server_key_iv: "d28ab4e7d0400d4f450f5cd7d490e20a"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.205178 15866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:25.224896 15866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:25.225198 15866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:25.225325 15866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:25.228997 15866 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:37905
I20260924 07:14:25.229033 15932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:37905 every 8 connection(s)
I20260924 07:14:25.229351 15866 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:25.230206 15933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:25.232481 15933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db: Bootstrap starting.
I20260924 07:14:25.233034 15933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:25.233294 15933 log.cc:824] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:25.233888 15933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db: No bootstrap required, opened a new log
I20260924 07:14:25.235395 15933 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ffb7e2a4ef4486af62f99a1c8b72db" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37905 } }
I20260924 07:14:25.235525 15933 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:25.235555 15933 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ffb7e2a4ef4486af62f99a1c8b72db, State: Initialized, Role: FOLLOWER
I20260924 07:14:25.235715 15933 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ffb7e2a4ef4486af62f99a1c8b72db" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37905 } }
I20260924 07:14:25.235793 15933 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:25.235827 15933 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:25.235873 15933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:25.236369 15933 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ffb7e2a4ef4486af62f99a1c8b72db" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37905 } }
I20260924 07:14:25.236456 15933 leader_election.cc:302] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6ffb7e2a4ef4486af62f99a1c8b72db; no voters:
I20260924 07:14:25.236603 15933 leader_election.cc:288] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:25.236639 15936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:25.236783 15936 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [term 1 LEADER]: Becoming Leader. State: Replica: a6ffb7e2a4ef4486af62f99a1c8b72db, State: Running, Role: LEADER
I20260924 07:14:25.236829 15933 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:25.236889 15936 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ffb7e2a4ef4486af62f99a1c8b72db" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37905 } }
I20260924 07:14:25.237216 15938 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6ffb7e2a4ef4486af62f99a1c8b72db. Latest consensus state: current_term: 1 leader_uuid: "a6ffb7e2a4ef4486af62f99a1c8b72db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ffb7e2a4ef4486af62f99a1c8b72db" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37905 } } }
I20260924 07:14:25.237295 15937 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6ffb7e2a4ef4486af62f99a1c8b72db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6ffb7e2a4ef4486af62f99a1c8b72db" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37905 } } }
I20260924 07:14:25.237308 15938 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:25.237481 15937 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:25.237722 15945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:25.238219 15945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:25.238247 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 15866
I20260924 07:14:25.238315 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:25.238497 10763 external_mini_cluster.cc:1428] Setting key 906e309933254a23833a9d22fe8be8a3
I20260924 07:14:25.239635 15945 catalog_manager.cc:1347] Generated new cluster ID: 06d8bf7feb8349509177ad3bb33d3b2b
I20260924 07:14:25.239689 15945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:25.244163 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.238894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55914 (local address 127.10.130.254:37905)
0924 07:14:25.239086 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.239095 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.240174 (+ 1079us) server_negotiation.cc:407] Connection header received
0924 07:14:25.240303 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.240310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.240370 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.240482 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:25.240681 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.241598 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.242360 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.242802 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.242908 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:25.243521 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:25.243525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:25.243525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:25.243691 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:25.243724 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:25.243736 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:25.243795 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:25.243865 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 07:14:25.243941 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
I20260924 07:14:25.245182 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:14:25.251910 15945 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:25.252604 15945 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:25.259645 15945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6ffb7e2a4ef4486af62f99a1c8b72db: Generated new TSK 0
I20260924 07:14:25.259826 15945 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 07:14:25.282759 15963 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260924 07:14:25.282845 15963 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 07:14:25.283123 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.268035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55920 (local address 127.10.130.254:37905)
0924 07:14:25.268076 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.268078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.268229 (+ 151us) server_negotiation.cc:407] Connection header received
0924 07:14:25.268305 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.268306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.268317 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.268415 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0924 07:14:25.268507 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.269274 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.269974 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.270289 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.270357 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:25.270495 (+ 138us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 07:14:25.282909 (+ 12414us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 07:14:25.283050 (+ 141us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55920: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 07:14:25.283190 15955 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55920: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260924 07:14:25.283408 15962 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:25.267969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:37905 (local address 127.0.0.1:55920)
0924 07:14:25.268167 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:25.268177 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:25.268231 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:25.268414 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:25.268415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:25.268416 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0924 07:14:25.268468 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:25.268471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.269350 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.269350 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:25.269904 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:25.269906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.270353 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.270354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:25.270392 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:25.270408 (+ 16us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 07:14:25.283014 (+ 12606us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 07:14:25.283103 (+ 89us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.130.254:37905: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":83,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.10.130.254:37905: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260924 07:14:25.283830 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 15866
2026-09-24T07:14:25Z chronyd exiting
I20260924 07:14:25.318572 10763 test_util.cc:175] -----------------------------------------------
I20260924 07:14:25.318651 10763 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790234044967003-10763-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260924 07:14:25.319371 10763 mini_oidc.cc:152] Starting JWKS server
I20260924 07:14:25.320037 10763 webserver.cc:460] Webserver started at https://127.0.0.1:35381/ using document root <none> and password file <none>
I20260924 07:14:25.320083 10763 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 07:14:25.320217 10763 webserver.cc:460] Webserver started at http://127.0.0.1:42511/ using document root <none> and password file <none>
2026-09-24T07:14:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:25Z Disabled control of system clock
I20260924 07:14:25.327116 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39133
--webserver_interface=127.10.130.254
--webserver_port=0
--builtin_ntp_servers=127.10.130.212:36907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:39133
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35381/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/testchainca.cert with env {}
W20260924 07:14:25.404141 15969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:25.404336 15969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:25.404356 15969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:25.405483 15969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:25.405521 15969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:25.405534 15969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:25.405544 15969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:25.405566 15969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 07:14:25.405582 15969 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 07:14:25.405601 15969 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35381/jwks.json
I20260924 07:14:25.407120 15969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:36907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:39133
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35381/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:25.407356 15969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:25.407543 15969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:25.411921 15975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:25.412006 15969 server_base.cc:1034] running on GCE node
W20260924 07:14:25.411932 15974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:25.411926 15977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:25.412354 15969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:25.412626 15969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:25.413756 15969 hybrid_clock.cc:648] HybridClock initialized: now 1790234065413743 us; error 26 us; skew 500 ppm
I20260924 07:14:25.414872 15969 webserver.cc:460] Webserver started at http://127.10.130.254:45301/ using document root <none> and password file <none>
I20260924 07:14:25.415074 15969 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:25.415117 15969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:25.415233 15969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:25.416265 15969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "abac989937fb4873a3f4d82d6cdbe0c2"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "6bec049ddb80337eb5cf64c1db039277"
server_key_iv: "171936d690ffcf3602c7d9da1c553c6b"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.416581 15969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "abac989937fb4873a3f4d82d6cdbe0c2"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "6bec049ddb80337eb5cf64c1db039277"
server_key_iv: "171936d690ffcf3602c7d9da1c553c6b"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.417713 15969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:25.418313 15983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:25.418437 15969 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:25.418514 15969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "abac989937fb4873a3f4d82d6cdbe0c2"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "6bec049ddb80337eb5cf64c1db039277"
server_key_iv: "171936d690ffcf3602c7d9da1c553c6b"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.418570 15969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:25.434280 15969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:25.434526 15969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:25.434643 15969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:25.438144 15969 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:39133
I20260924 07:14:25.438181 16035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:39133 every 8 connection(s)
I20260924 07:14:25.438464 15969 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:25.439047 16036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:25.440655 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 15969
I20260924 07:14:25.440735 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:25.440881 10763 external_mini_cluster.cc:1428] Setting key 41c62eb7f1aa19549fe54eebf129b85d
I20260924 07:14:25.442158 16036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2: Bootstrap starting.
I20260924 07:14:25.442845 16036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:25.443163 16036 log.cc:824] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:25.444039 16036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2: No bootstrap required, opened a new log
I20260924 07:14:25.446194 16036 raft_consensus.cc:348] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abac989937fb4873a3f4d82d6cdbe0c2" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39133 } }
I20260924 07:14:25.446363 16036 raft_consensus.cc:374] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:25.446437 16036 raft_consensus.cc:729] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abac989937fb4873a3f4d82d6cdbe0c2, State: Initialized, Role: FOLLOWER
I20260924 07:14:25.446614 16039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.441312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38916 (local address 127.10.130.254:39133)
0924 07:14:25.441491 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.441500 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.441533 (+ 33us) server_negotiation.cc:407] Connection header received
0924 07:14:25.442038 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.442047 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.442125 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.442276 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:25.442465 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.443761 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.444600 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.445186 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.445319 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:25.445898 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:25.445902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:25.445903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:25.446105 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:25.446149 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:25.446164 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:25.446240 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:25.446284 (+ 44us) server_negotiation.cc:299] Negotiation successful
0924 07:14:25.446359 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
I20260924 07:14:25.446574 16036 consensus_queue.cc:260] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abac989937fb4873a3f4d82d6cdbe0c2" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39133 } }
I20260924 07:14:25.446677 16036 raft_consensus.cc:388] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:25.446712 16036 raft_consensus.cc:482] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:25.446762 16036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:25.447391 16036 raft_consensus.cc:504] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abac989937fb4873a3f4d82d6cdbe0c2" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39133 } }
I20260924 07:14:25.447501 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:14:25.447500 16036 leader_election.cc:302] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abac989937fb4873a3f4d82d6cdbe0c2; no voters:
I20260924 07:14:25.447649 16036 leader_election.cc:288] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:25.447672 16041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:25.447813 16041 raft_consensus.cc:686] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [term 1 LEADER]: Becoming Leader. State: Replica: abac989937fb4873a3f4d82d6cdbe0c2, State: Running, Role: LEADER
I20260924 07:14:25.447870 16036 sys_catalog.cc:564] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:25.447894 16041 consensus_queue.cc:237] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abac989937fb4873a3f4d82d6cdbe0c2" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39133 } }
I20260924 07:14:25.448340 16042 sys_catalog.cc:455] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abac989937fb4873a3f4d82d6cdbe0c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abac989937fb4873a3f4d82d6cdbe0c2" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39133 } } }
I20260924 07:14:25.448422 16042 sys_catalog.cc:458] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:25.448582 16043 sys_catalog.cc:455] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abac989937fb4873a3f4d82d6cdbe0c2. Latest consensus state: current_term: 1 leader_uuid: "abac989937fb4873a3f4d82d6cdbe0c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abac989937fb4873a3f4d82d6cdbe0c2" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39133 } } }
I20260924 07:14:25.448624 16049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:25.448642 16043 sys_catalog.cc:458] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:25.449177 16049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:25.450624 16049 catalog_manager.cc:1347] Generated new cluster ID: d0e110eb45d6400f91bebfb31c45114c
I20260924 07:14:25.450675 16049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:25.466030 16049 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:25.466706 16049 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:25.472172 16049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abac989937fb4873a3f4d82d6cdbe0c2: Generated new TSK 0
I20260924 07:14:25.472370 16049 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 07:14:25.493111 16066 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260924 07:14:25.493175 16066 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 07:14:25.493515 16039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.479725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38918 (local address 127.10.130.254:39133)
0924 07:14:25.479770 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.479772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.479839 (+ 67us) server_negotiation.cc:407] Connection header received
0924 07:14:25.479896 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.479896 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.479908 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.479995 (+ 87us) server_negotiation.cc:226] Negotiated authn=JWT
0924 07:14:25.480114 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.480893 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.481582 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.481884 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.481949 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:25.482053 (+ 104us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 07:14:25.493312 (+ 11259us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 07:14:25.493444 (+ 132us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38918: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260924 07:14:25.493554 16065 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:25.479646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39133 (local address 127.0.0.1:38918)
0924 07:14:25.479787 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:25.479797 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:25.479848 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:25.480004 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:25.480005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:25.480007 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 07:14:25.480062 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:25.480065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.480959 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.480960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:25.481519 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:25.481521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.481950 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.481950 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:25.481987 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:25.481991 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 07:14:25.493402 (+ 11411us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 07:14:25.493485 (+ 83us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.130.254:39133: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
W20260924 07:14:25.493568 16039 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38918: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260924 07:14:25.493928 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 15969
2026-09-24T07:14:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (211 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260924 07:14:25.530542 10763 mini_oidc.cc:152] Starting JWKS server
I20260924 07:14:25.531190 10763 webserver.cc:460] Webserver started at https://127.0.0.1:33177/ using document root <none> and password file <none>
I20260924 07:14:25.531234 10763 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 07:14:25.531368 10763 webserver.cc:460] Webserver started at http://127.0.0.1:37089/ using document root <none> and password file <none>
2026-09-24T07:14:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:25Z Disabled control of system clock
I20260924 07:14:25.538261 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:34265
--webserver_interface=127.10.130.254
--webserver_port=0
--builtin_ntp_servers=127.10.130.212:42733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:34265
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33177/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 07:14:25.614246 16072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:25.614434 16072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:25.614452 16072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:25.615537 16072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:25.615569 16072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:25.615581 16072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:25.615592 16072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:25.615602 16072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 07:14:25.615612 16072 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 07:14:25.615622 16072 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33177/jwks.json
I20260924 07:14:25.616952 16072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:42733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:34265
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:34265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33177/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:25.617141 16072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:25.617358 16072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:25.621702 16077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:25.621702 16080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:25.621889 16072 server_base.cc:1034] running on GCE node
W20260924 07:14:25.621923 16078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:25.622157 16072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:25.622431 16072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:25.623566 16072 hybrid_clock.cc:648] HybridClock initialized: now 1790234065623548 us; error 36 us; skew 500 ppm
I20260924 07:14:25.624756 16072 webserver.cc:460] Webserver started at http://127.10.130.254:38021/ using document root <none> and password file <none>
I20260924 07:14:25.624980 16072 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:25.625023 16072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:25.625144 16072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:25.626137 16072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "ae1a24d932214d2a974297b707b3f178"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "4c7eb9a37d82e0d120841855a6140300"
server_key_iv: "d53497cd27e54b3b7573c86465429b2a"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.626462 16072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "ae1a24d932214d2a974297b707b3f178"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "4c7eb9a37d82e0d120841855a6140300"
server_key_iv: "d53497cd27e54b3b7573c86465429b2a"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.627617 16072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:14:25.628260 16086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:25.628388 16072 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:25.628466 16072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "ae1a24d932214d2a974297b707b3f178"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "4c7eb9a37d82e0d120841855a6140300"
server_key_iv: "d53497cd27e54b3b7573c86465429b2a"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.628530 16072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:25.647969 16072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:25.648262 16072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:25.648384 16072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:25.651782 16072 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:34265
I20260924 07:14:25.651816 16138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:34265 every 8 connection(s)
I20260924 07:14:25.652114 16072 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:25.652684 16139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:25.655197 16139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178: Bootstrap starting.
I20260924 07:14:25.655678 16139 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:25.655890 16139 log.cc:824] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:25.656478 16139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178: No bootstrap required, opened a new log
I20260924 07:14:25.657945 16139 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1a24d932214d2a974297b707b3f178" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34265 } }
I20260924 07:14:25.658074 16139 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:25.658095 16139 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae1a24d932214d2a974297b707b3f178, State: Initialized, Role: FOLLOWER
I20260924 07:14:25.658213 16139 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1a24d932214d2a974297b707b3f178" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34265 } }
I20260924 07:14:25.658288 16139 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:25.658317 16139 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:25.658349 16139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:25.658805 16139 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1a24d932214d2a974297b707b3f178" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34265 } }
I20260924 07:14:25.658885 16139 leader_election.cc:302] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae1a24d932214d2a974297b707b3f178; no voters:
I20260924 07:14:25.659005 16139 leader_election.cc:288] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:25.659054 16142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:25.659188 16142 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [term 1 LEADER]: Becoming Leader. State: Replica: ae1a24d932214d2a974297b707b3f178, State: Running, Role: LEADER
I20260924 07:14:25.659227 16139 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:25.659278 16142 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1a24d932214d2a974297b707b3f178" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34265 } }
I20260924 07:14:25.659596 16143 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae1a24d932214d2a974297b707b3f178" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1a24d932214d2a974297b707b3f178" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34265 } } }
I20260924 07:14:25.659721 16143 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:25.659770 16144 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae1a24d932214d2a974297b707b3f178. Latest consensus state: current_term: 1 leader_uuid: "ae1a24d932214d2a974297b707b3f178" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae1a24d932214d2a974297b707b3f178" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34265 } } }
I20260924 07:14:25.659827 16144 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:25.660175 16151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:25.660609 16151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:25.661772 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 16072
I20260924 07:14:25.661852 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:25.662027 10763 external_mini_cluster.cc:1428] Setting key 6654938957a8cafb0aae327f8c3e292a
I20260924 07:14:25.662065 16151 catalog_manager.cc:1347] Generated new cluster ID: 9bc1510a3a854d1b96f055ed66c2d1db
I20260924 07:14:25.662103 16151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:25.672189 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.664071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.10.130.254:34265)
0924 07:14:25.668092 (+ 4021us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.668099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.668124 (+ 25us) server_negotiation.cc:407] Connection header received
0924 07:14:25.668217 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.668222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.668273 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.668394 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:25.668600 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.669713 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.670459 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.670863 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.670966 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:25.671561 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:25.671565 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:25.671566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:25.671737 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:25.671767 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:25.671778 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:25.671830 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:25.671909 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 07:14:25.672002 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3959,"thread_start_us":54,"threads_started":1}
I20260924 07:14:25.672904 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:14:25.676152 16151 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:25.676600 16151 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:25.679502 16151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae1a24d932214d2a974297b707b3f178: Generated new TSK 0
I20260924 07:14:25.679648 16151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:14:25.688385 16162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.685398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52522 (local address 127.10.130.254:34265)
0924 07:14:25.685438 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.685439 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.685548 (+ 109us) server_negotiation.cc:407] Connection header received
0924 07:14:25.685616 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.685617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.685627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.685679 (+ 52us) server_negotiation.cc:226] Negotiated authn=JWT
0924 07:14:25.685791 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.686249 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.686980 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.687178 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.687228 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:25.687389 (+ 161us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 07:14:25.688183 (+ 794us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33177/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 07:14:25.688307 (+ 124us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52522: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33177/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":23}
W20260924 07:14:25.688431 16162 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52522: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33177/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260924 07:14:25.688477 16168 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:25.685350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34265 (local address 127.0.0.1:52522)
0924 07:14:25.685490 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:25.685500 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:25.685554 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:25.685682 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:25.685683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:25.685684 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0924 07:14:25.685738 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:25.685739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.686336 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.686337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:25.686912 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:25.686914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.687238 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.687239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:25.687295 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:25.687300 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 07:14:25.688276 (+ 976us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33177/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 07:14:25.688378 (+ 102us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.130.254:34265: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33177/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260924 07:14:25.688614 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 16072
E20260924 07:14:25.700676 16169 webserver.cc:533] Webserver: sslize failed:
E20260924 07:14:25.700743 16169 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-24T07:14:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (195 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-24T07:14:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:25Z Disabled control of system clock
I20260924 07:14:25.732810 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:45951
--webserver_interface=127.10.130.254
--webserver_port=0
--builtin_ntp_servers=127.10.130.212:34063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:45951
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/testchainca.cert with env {}
W20260924 07:14:25.809276 16173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:25.809465 16173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:25.809485 16173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:25.810642 16173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:25.810683 16173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:25.810693 16173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:25.810707 16173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:25.810730 16173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 07:14:25.810752 16173 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 07:14:25.810772 16173 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260924 07:14:25.812247 16173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:34063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:45951
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:45951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:25.812469 16173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:25.812662 16173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:25.817000 16179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:25.817016 16181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:25.817209 16173 server_base.cc:1034] running on GCE node
W20260924 07:14:25.817219 16178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:25.817476 16173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:25.817732 16173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:25.818877 16173 hybrid_clock.cc:648] HybridClock initialized: now 1790234065818860 us; error 25 us; skew 500 ppm
I20260924 07:14:25.820186 16173 webserver.cc:460] Webserver started at http://127.10.130.254:36273/ using document root <none> and password file <none>
I20260924 07:14:25.820415 16173 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:25.820467 16173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:25.820598 16173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:25.821666 16173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "ec04f0e0d02f44428f1a7ac5225a138b"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "5063ca7ecdeb2316611b64850e3a3b50"
server_key_iv: "61013877b57452fcbbadc5dbc410b6a8"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.821995 16173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "ec04f0e0d02f44428f1a7ac5225a138b"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "5063ca7ecdeb2316611b64850e3a3b50"
server_key_iv: "61013877b57452fcbbadc5dbc410b6a8"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.823190 16173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:14:25.823838 16187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:25.824024 16173 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:25.824121 16173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "ec04f0e0d02f44428f1a7ac5225a138b"
format_stamp: "Formatted at 2026-09-24 07:14:25 on dist-test-slave-7q7m"
server_key: "5063ca7ecdeb2316611b64850e3a3b50"
server_key_iv: "61013877b57452fcbbadc5dbc410b6a8"
server_key_version: "encryptionkey@0"
I20260924 07:14:25.824193 16173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:25.845034 16173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:25.845368 16173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:25.845508 16173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:25.849462 16173 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:45951
I20260924 07:14:25.849509 16239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:45951 every 8 connection(s)
I20260924 07:14:25.849839 16173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:25.850735 16240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:25.852957 16240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b: Bootstrap starting.
I20260924 07:14:25.853474 16240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:25.853703 16240 log.cc:824] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:25.854293 16240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b: No bootstrap required, opened a new log
I20260924 07:14:25.855765 16240 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec04f0e0d02f44428f1a7ac5225a138b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 45951 } }
I20260924 07:14:25.855897 16240 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:25.855918 16240 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec04f0e0d02f44428f1a7ac5225a138b, State: Initialized, Role: FOLLOWER
I20260924 07:14:25.856055 16240 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec04f0e0d02f44428f1a7ac5225a138b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 45951 } }
I20260924 07:14:25.856128 16240 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:25.856158 16240 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:25.856190 16240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:25.856354 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 16173
I20260924 07:14:25.856418 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:25.856550 10763 external_mini_cluster.cc:1428] Setting key 7a49e054e7c1093c4b314eaf2410117a
I20260924 07:14:25.856635 16240 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec04f0e0d02f44428f1a7ac5225a138b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 45951 } }
I20260924 07:14:25.856707 16240 leader_election.cc:302] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec04f0e0d02f44428f1a7ac5225a138b; no voters:
I20260924 07:14:25.856863 16240 leader_election.cc:288] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:25.856916 16243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:25.857076 16243 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [term 1 LEADER]: Becoming Leader. State: Replica: ec04f0e0d02f44428f1a7ac5225a138b, State: Running, Role: LEADER
I20260924 07:14:25.857138 16240 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:25.857209 16243 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec04f0e0d02f44428f1a7ac5225a138b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 45951 } }
I20260924 07:14:25.857564 16243 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec04f0e0d02f44428f1a7ac5225a138b. Latest consensus state: current_term: 1 leader_uuid: "ec04f0e0d02f44428f1a7ac5225a138b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec04f0e0d02f44428f1a7ac5225a138b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 45951 } } }
I20260924 07:14:25.857658 16243 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:25.857574 16247 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec04f0e0d02f44428f1a7ac5225a138b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec04f0e0d02f44428f1a7ac5225a138b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 45951 } } }
I20260924 07:14:25.857771 16247 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:25.857904 16249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:25.858562 16249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:25.860013 16249 catalog_manager.cc:1347] Generated new cluster ID: c25302728f1f44419fd61f9590a92939
I20260924 07:14:25.860067 16249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:25.862147 16246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.856967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54564 (local address 127.10.130.254:45951)
0924 07:14:25.857181 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.857190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.857223 (+ 33us) server_negotiation.cc:407] Connection header received
0924 07:14:25.857340 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.857348 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.857421 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.857827 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:25.857929 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.859243 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.860116 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.860699 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.860806 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:25.861483 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:25.861488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:25.861488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:25.861688 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:25.861726 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:25.861739 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:25.861794 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:25.861882 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 07:14:25.861954 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":61,"threads_started":1}
I20260924 07:14:25.863000 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:14:25.871462 16249 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:25.871950 16249 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:25.877920 16249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec04f0e0d02f44428f1a7ac5225a138b: Generated new TSK 0
I20260924 07:14:25.878055 16249 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:14:26.309796 16246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:25.885302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54572 (local address 127.10.130.254:45951)
0924 07:14:25.885359 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:25.885361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:25.885455 (+ 94us) server_negotiation.cc:407] Connection header received
0924 07:14:25.885513 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:25.885514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:25.885523 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:25.885577 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0924 07:14:25.885734 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.886210 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.887273 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.887464 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.887506 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:25.887669 (+ 163us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 07:14:26.309569 (+421900us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 07:14:26.309726 (+ 157us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54572: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":34}
W20260924 07:14:26.309862 16246 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54572: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260924 07:14:26.310009 16269 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:25.885239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:45951 (local address 127.0.0.1:54572)
0924 07:14:25.885388 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:25.885398 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:25.885459 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:25.885586 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:25.885586 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:25.885588 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 07:14:25.885680 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:25.885682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.886270 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.886271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:25.887204 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:25.887206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:25.887518 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:25.887519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:25.887582 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:25.887587 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 07:14:26.309747 (+422160us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 07:14:26.309892 (+ 145us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.130.254:45951: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260924 07:14:26.310325 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 16173
2026-09-24T07:14:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (601 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (84 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (86 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:26 dist-test-slave-7q7m krb5kdc[16281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:26 dist-test-slave-7q7m krb5kdc[16281](info): set up 2 sockets
Sep 24 07:14:26 dist-test-slave-7q7m krb5kdc[16281](info): commencing operation
krb5kdc: starting...
W20260924 07:14:27.419793 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.921s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:27 dist-test-slave-7q7m krb5kdc[16281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:14:27.521826 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:42495
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:42495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:27.598209 16294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:27.598383 16294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:27.598402 16294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:27.599483 16294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:27.599512 16294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:27.599524 16294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:27.599535 16294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:27.599545 16294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:27.600929 16294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:42495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:42495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:27.601104 16294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:27.601265 16294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:27.605654 16299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:27.605665 16302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:27.605631 16300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:27.605738 16294 server_base.cc:1034] running on GCE node
I20260924 07:14:27.606060 16294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:27.606334 16294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:27.607471 16294 hybrid_clock.cc:648] HybridClock initialized: now 1790234067607457 us; error 29 us; skew 500 ppm
Sep 24 07:14:27 dist-test-slave-7q7m krb5kdc[16281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234067, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:27.613075 16294 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:14:27.613452 16294 webserver.cc:460] Webserver started at http://127.10.130.254:45529/ using document root <none> and password file <none>
I20260924 07:14:27.613662 16294 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:27.613708 16294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:27.613833 16294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:27.614804 16294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "f67d354ec6d44441a3f05ad4b85178ca"
format_stamp: "Formatted at 2026-09-24 07:14:27 on dist-test-slave-7q7m"
server_key: "5e0a955c5803b50e8fa22d98e8f62391"
server_key_iv: "7ad992514f48936d824f4411276bfdab"
server_key_version: "encryptionkey@0"
I20260924 07:14:27.615108 16294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "f67d354ec6d44441a3f05ad4b85178ca"
format_stamp: "Formatted at 2026-09-24 07:14:27 on dist-test-slave-7q7m"
server_key: "5e0a955c5803b50e8fa22d98e8f62391"
server_key_iv: "7ad992514f48936d824f4411276bfdab"
server_key_version: "encryptionkey@0"
I20260924 07:14:27.616331 16294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:14:27.616987 16309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:27.617127 16294 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:27.617204 16294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "f67d354ec6d44441a3f05ad4b85178ca"
format_stamp: "Formatted at 2026-09-24 07:14:27 on dist-test-slave-7q7m"
server_key: "5e0a955c5803b50e8fa22d98e8f62391"
server_key_iv: "7ad992514f48936d824f4411276bfdab"
server_key_version: "encryptionkey@0"
I20260924 07:14:27.617261 16294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:27.626904 16294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:27.628845 16294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:27.628969 16294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:27.632444 16294 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:42495
I20260924 07:14:27.632474 16361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:42495 every 8 connection(s)
I20260924 07:14:27.632761 16294 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:27.633505 16362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:27.635262 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 16294
I20260924 07:14:27.635346 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:27.635502 10763 external_mini_cluster.cc:1428] Setting key 7420bf7672299f24a58807b2c2dc09bb
I20260924 07:14:27.635613 16362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca: Bootstrap starting.
I20260924 07:14:27.636270 16362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:27.636533 16362 log.cc:824] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:27.637146 16362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca: No bootstrap required, opened a new log
I20260924 07:14:27.639292 16362 raft_consensus.cc:348] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67d354ec6d44441a3f05ad4b85178ca" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42495 } }
I20260924 07:14:27.639434 16362 raft_consensus.cc:374] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:27.639468 16362 raft_consensus.cc:729] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f67d354ec6d44441a3f05ad4b85178ca, State: Initialized, Role: FOLLOWER
I20260924 07:14:27.639580 16362 consensus_queue.cc:260] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67d354ec6d44441a3f05ad4b85178ca" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42495 } }
I20260924 07:14:27.639644 16362 raft_consensus.cc:388] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:27.639678 16362 raft_consensus.cc:482] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:27.639717 16362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:27.640373 16362 raft_consensus.cc:504] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67d354ec6d44441a3f05ad4b85178ca" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42495 } }
I20260924 07:14:27.640476 16362 leader_election.cc:302] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f67d354ec6d44441a3f05ad4b85178ca; no voters:
I20260924 07:14:27.640638 16362 leader_election.cc:288] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:27.640692 16367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:27.640833 16367 raft_consensus.cc:686] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [term 1 LEADER]: Becoming Leader. State: Replica: f67d354ec6d44441a3f05ad4b85178ca, State: Running, Role: LEADER
I20260924 07:14:27.640883 16362 sys_catalog.cc:564] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:27.640985 16367 consensus_queue.cc:237] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67d354ec6d44441a3f05ad4b85178ca" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42495 } }
I20260924 07:14:27.641314 16367 sys_catalog.cc:455] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader f67d354ec6d44441a3f05ad4b85178ca. Latest consensus state: current_term: 1 leader_uuid: "f67d354ec6d44441a3f05ad4b85178ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67d354ec6d44441a3f05ad4b85178ca" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42495 } } }
I20260924 07:14:27.641507 16367 sys_catalog.cc:458] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:27.641448 16368 sys_catalog.cc:455] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f67d354ec6d44441a3f05ad4b85178ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f67d354ec6d44441a3f05ad4b85178ca" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42495 } } }
I20260924 07:14:27.641741 16368 sys_catalog.cc:458] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca [sys.catalog]: This master's current role is: LEADER
Sep 24 07:14:27 dist-test-slave-7q7m krb5kdc[16281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234067, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
W20260924 07:14:27.642580 16383 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:14:27.642669 16383 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:14:27.642762 16376 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:27.642915 16376 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:27.644320 16376 catalog_manager.cc:1347] Generated new cluster ID: 4d65336e46f748e5830ab59ae2993fe4
I20260924 07:14:27.644368 16376 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:27.646136 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:27.635925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37456 (local address 127.10.130.254:42495)
0924 07:14:27.636096 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:27.636106 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:27.636132 (+ 26us) server_negotiation.cc:407] Connection header received
0924 07:14:27.636574 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:27.636582 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:27.636642 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:27.636754 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:27.637371 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:27.638703 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:27.639405 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:27.639996 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:27.640107 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:27.642564 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:27.642569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:27.642569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:27.642580 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:27.644867 (+ 2287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:27.645158 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:27.645161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:27.645161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:27.645241 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:27.645401 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:27.645402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:27.645402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:27.645613 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:27.645752 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:27.645862 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 07:14:27.645984 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260924 07:14:27.667271 16376 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:27.667874 16376 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:27.675554 16376 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f67d354ec6d44441a3f05ad4b85178ca: Generated new TSK 0
I20260924 07:14:27.675726 16376 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:14:27.710462 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42495
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:27.812068 16388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:27.812274 16388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:27.812309 16388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:27.813472 16388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:27.813515 16388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:27.813555 16388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:27.815086 16388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:27.815321 16388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:27.815527 16388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:27.820252 16396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:27.820238 16394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:27.820282 16388 server_base.cc:1034] running on GCE node
W20260924 07:14:27.820238 16393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:27.820691 16388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:27.820959 16388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:27.822094 16388 hybrid_clock.cc:648] HybridClock initialized: now 1790234067822082 us; error 28 us; skew 500 ppm
Sep 24 07:14:27 dist-test-slave-7q7m krb5kdc[16281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234067, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:27.824553 16388 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:14:27.824980 16388 webserver.cc:460] Webserver started at http://127.10.130.193:39659/ using document root <none> and password file <none>
I20260924 07:14:27.825187 16388 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:27.825232 16388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:27.825348 16388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:27.826363 16388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "119cea4e607a4ef8b4e01370a60f4b68"
format_stamp: "Formatted at 2026-09-24 07:14:27 on dist-test-slave-7q7m"
server_key: "6d58b04dcf82347b03c082c7853ac02c"
server_key_iv: "b9cfc6a8dabd170fd0b595cfe7dec566"
server_key_version: "encryptionkey@0"
I20260924 07:14:27.826675 16388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "119cea4e607a4ef8b4e01370a60f4b68"
format_stamp: "Formatted at 2026-09-24 07:14:27 on dist-test-slave-7q7m"
server_key: "6d58b04dcf82347b03c082c7853ac02c"
server_key_iv: "b9cfc6a8dabd170fd0b595cfe7dec566"
server_key_version: "encryptionkey@0"
I20260924 07:14:27.827824 16388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:27.828470 16403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:27.828595 16388 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:27.828693 16388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "119cea4e607a4ef8b4e01370a60f4b68"
format_stamp: "Formatted at 2026-09-24 07:14:27 on dist-test-slave-7q7m"
server_key: "6d58b04dcf82347b03c082c7853ac02c"
server_key_iv: "b9cfc6a8dabd170fd0b595cfe7dec566"
server_key_version: "encryptionkey@0"
I20260924 07:14:27.828753 16388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:27.848568 16388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:27.850666 16388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:27.850802 16388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:27.851008 16388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:27.851291 16388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:27.851323 16388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:27.851352 16388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:27.851374 16388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:27.856729 16388 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:41725
I20260924 07:14:27.856792 16516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:41725 every 8 connection(s)
I20260924 07:14:27.857071 16388 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
Sep 24 07:14:27 dist-test-slave-7q7m krb5kdc[16281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234067, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:27.865489 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 16388
I20260924 07:14:27.865592 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:27.865675 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:27.857763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:47509 (local address 127.10.130.254:42495)
0924 07:14:27.857819 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:27.857820 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:27.858015 (+ 195us) server_negotiation.cc:407] Connection header received
0924 07:14:27.858878 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:27.858879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:27.858887 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:27.858942 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:27.859524 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:27.860090 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:27.861086 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:27.861277 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:27.861320 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:27.863213 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:27.863214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:27.863214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:27.863221 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:27.864447 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:27.864820 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:27.864821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:27.864821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:27.864867 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:27.865072 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:27.865073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:27.865073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:27.865179 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:27.865227 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:27.865509 (+ 282us) server_negotiation.cc:299] Negotiation successful
0924 07:14:27.865583 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 07:14:27.865794 10763 external_mini_cluster.cc:1428] Setting key 47729a67e5a81e5129eaa8edaf10ea06
I20260924 07:14:27.865953 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:27.857720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:42495 (local address 127.10.130.193:47509)
0924 07:14:27.857895 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:27.857937 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:27.858794 (+ 857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:27.858997 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:27.859000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:27.859241 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:27.859427 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:27.859431 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:27.860187 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:27.860188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:27.861016 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:27.861019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:27.861330 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:27.861330 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:27.861430 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:27.861851 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:27.861859 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:27.863131 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:27.864527 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:27.864532 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:27.864534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:27.864762 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:27.864913 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:27.864913 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:27.864914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:27.865026 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:27.865225 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:27.865227 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:27.865384 (+ 157us) client_negotiation.cc:769] Sending connection context
0924 07:14:27.865490 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 07:14:27.865673 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260924 07:14:27.866394 16517 heartbeater.cc:347] Connected to a master server at 127.10.130.254:42495
I20260924 07:14:27.866492 16517 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:27.866683 16517 heartbeater.cc:511] Master 127.10.130.254:42495 requested a full tablet report, sending...
I20260924 07:14:27.867170 16326 ts_manager.cc:194] Registered new tserver with Master: 119cea4e607a4ef8b4e01370a60f4b68 (127.10.130.193:41725)
I20260924 07:14:27.867758 16326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:47509
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:14:27.885990 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42495
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:27.962534 16522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:27.962718 16522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:27.962746 16522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:27.963840 16522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:27.963876 16522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:27.963910 16522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:27.965304 16522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:27.965512 16522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:27.965708 16522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:27.970237 16528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:27.970212 16527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:27.970221 16530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:27.970309 16522 server_base.cc:1034] running on GCE node
I20260924 07:14:27.970571 16522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:27.970762 16522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:27.971894 16522 hybrid_clock.cc:648] HybridClock initialized: now 1790234067971879 us; error 25 us; skew 500 ppm
Sep 24 07:14:27 dist-test-slave-7q7m krb5kdc[16281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234067, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:27.974088 16522 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:14:27.974440 16522 webserver.cc:460] Webserver started at http://127.10.130.194:41019/ using document root <none> and password file <none>
I20260924 07:14:27.974614 16522 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:27.974666 16522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:27.974781 16522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:27.975711 16522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "62ac70def2484a079a5dd44fea1f0347"
format_stamp: "Formatted at 2026-09-24 07:14:27 on dist-test-slave-7q7m"
server_key: "9241d920a5598d572a2ed90afe49aafd"
server_key_iv: "0dfae9be51f9b1467d53ca2865d9d9d0"
server_key_version: "encryptionkey@0"
I20260924 07:14:27.976052 16522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "62ac70def2484a079a5dd44fea1f0347"
format_stamp: "Formatted at 2026-09-24 07:14:27 on dist-test-slave-7q7m"
server_key: "9241d920a5598d572a2ed90afe49aafd"
server_key_iv: "0dfae9be51f9b1467d53ca2865d9d9d0"
server_key_version: "encryptionkey@0"
I20260924 07:14:27.977159 16522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 07:14:27.977775 16537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:27.977922 16522 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:27.977996 16522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "62ac70def2484a079a5dd44fea1f0347"
format_stamp: "Formatted at 2026-09-24 07:14:27 on dist-test-slave-7q7m"
server_key: "9241d920a5598d572a2ed90afe49aafd"
server_key_iv: "0dfae9be51f9b1467d53ca2865d9d9d0"
server_key_version: "encryptionkey@0"
I20260924 07:14:27.978057 16522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:27.995471 16522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:27.997061 16522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:27.997186 16522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:27.997380 16522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:27.997648 16522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:27.997680 16522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:27.997709 16522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:27.997725 16522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:28.003031 16522 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:34865
I20260924 07:14:28.003396 16522 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:14:28.004156 16650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:34865 every 8 connection(s)
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234067, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:28.009676 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 16522
I20260924 07:14:28.009768 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:28.009939 10763 external_mini_cluster.cc:1428] Setting key b86bf30a8f73a77d0004f320d46380d7
I20260924 07:14:28.012019 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:28.004164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:50593 (local address 127.10.130.254:42495)
0924 07:14:28.004206 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:28.004208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:28.004414 (+ 206us) server_negotiation.cc:407] Connection header received
0924 07:14:28.005274 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:28.005275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:28.005288 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:28.005339 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:28.005893 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.006379 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.007302 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.007512 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.007569 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:28.009409 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:28.009410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:28.009411 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:28.009418 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:28.010761 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:28.011096 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:28.011097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:28.011097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:28.011158 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:28.011358 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:28.011359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:28.011359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:28.011479 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:28.011544 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:28.011825 (+ 281us) server_negotiation.cc:299] Negotiation successful
0924 07:14:28.011904 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:14:28.012109 16653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:28.004143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:42495 (local address 127.10.130.194:50593)
0924 07:14:28.004292 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:28.004330 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:28.005167 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:28.005405 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:28.005409 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:28.005628 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:28.005801 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:28.005805 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.006455 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.006456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:28.007217 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:28.007219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.007578 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.007579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:28.007687 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:28.008077 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:28.008084 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:28.009324 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:28.010835 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:28.010841 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:28.010843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:28.011035 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:28.011210 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:28.011211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:28.011211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:28.011305 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:28.011550 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:28.011558 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:28.011703 (+ 145us) client_negotiation.cc:769] Sending connection context
0924 07:14:28.011795 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 07:14:28.011905 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260924 07:14:28.012524 16651 heartbeater.cc:347] Connected to a master server at 127.10.130.254:42495
I20260924 07:14:28.012599 16651 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:28.012749 16651 heartbeater.cc:511] Master 127.10.130.254:42495 requested a full tablet report, sending...
I20260924 07:14:28.013089 16326 ts_manager.cc:194] Registered new tserver with Master: 62ac70def2484a079a5dd44fea1f0347 (127.10.130.194:34865)
I20260924 07:14:28.013439 16326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:50593
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:14:28.030126 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42495
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:38393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:28.106716 16656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:28.106907 16656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:28.106937 16656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:28.108094 16656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:28.108139 16656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:28.108172 16656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:28.109566 16656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:42495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:28.109783 16656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:28.109984 16656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:28.114526 16661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:28.114521 16664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:28.114531 16662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:28.114737 16656 server_base.cc:1034] running on GCE node
I20260924 07:14:28.114935 16656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:28.115118 16656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:28.116255 16656 hybrid_clock.cc:648] HybridClock initialized: now 1790234068116245 us; error 25 us; skew 500 ppm
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234068, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:28.118649 16656 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:14:28.119056 16656 webserver.cc:460] Webserver started at http://127.10.130.195:44323/ using document root <none> and password file <none>
I20260924 07:14:28.119243 16656 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:28.119297 16656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:28.119408 16656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:28.120515 16656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "1da819ab6c5443678a1ebd1a6580c293"
format_stamp: "Formatted at 2026-09-24 07:14:28 on dist-test-slave-7q7m"
server_key: "daf75942baaf827a620752ebd740a3e0"
server_key_iv: "39b77d9d426e227339fad55c4f0864cd"
server_key_version: "encryptionkey@0"
I20260924 07:14:28.120842 16656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "1da819ab6c5443678a1ebd1a6580c293"
format_stamp: "Formatted at 2026-09-24 07:14:28 on dist-test-slave-7q7m"
server_key: "daf75942baaf827a620752ebd740a3e0"
server_key_iv: "39b77d9d426e227339fad55c4f0864cd"
server_key_version: "encryptionkey@0"
I20260924 07:14:28.122009 16656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:28.122671 16671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:28.122826 16656 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:28.122907 16656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "1da819ab6c5443678a1ebd1a6580c293"
format_stamp: "Formatted at 2026-09-24 07:14:28 on dist-test-slave-7q7m"
server_key: "daf75942baaf827a620752ebd740a3e0"
server_key_iv: "39b77d9d426e227339fad55c4f0864cd"
server_key_version: "encryptionkey@0"
I20260924 07:14:28.122965 16656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:28.135054 16656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:28.136670 16656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:28.136806 16656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:28.137006 16656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:28.137290 16656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:28.137322 16656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:28.137351 16656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:28.137368 16656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:28.142670 16656 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:37061
I20260924 07:14:28.142732 16784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:37061 every 8 connection(s)
I20260924 07:14:28.143011 16656 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:28.143869 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 16656
I20260924 07:14:28.143958 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:28.144184 10763 external_mini_cluster.cc:1428] Setting key f0dd73689085a850482d78c1fd6a89ca
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234068, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:28.151316 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:28.143665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:33359 (local address 127.10.130.254:42495)
0924 07:14:28.143715 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:28.143717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:28.143926 (+ 209us) server_negotiation.cc:407] Connection header received
0924 07:14:28.145015 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:28.145016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:28.145026 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:28.145062 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:28.145633 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.146155 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.147101 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.147287 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.147329 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:28.149272 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:28.149274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:28.149274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:28.149281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:28.150330 (+ 1049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:28.150677 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:28.150678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:28.150678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:28.150722 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:28.150882 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:28.150882 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:28.150883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:28.150971 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:28.151014 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:28.151204 (+ 190us) server_negotiation.cc:299] Negotiation successful
0924 07:14:28.151255 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:14:28.151504 16787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:28.143636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:42495 (local address 127.10.130.195:33359)
0924 07:14:28.143798 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:28.143837 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:28.144926 (+ 1089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:28.145122 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:28.145125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:28.145381 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:28.145562 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:28.145565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.146227 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.146228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:28.147034 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:28.147037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.147340 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.147341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:28.147444 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:28.147844 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:28.147852 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:28.149154 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:28.150401 (+ 1247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:28.150406 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:28.150408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:28.150625 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:28.150758 (+ 133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:28.150758 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:28.150759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:28.150843 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:28.151011 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:28.151014 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:28.151116 (+ 102us) client_negotiation.cc:769] Sending connection context
0924 07:14:28.151188 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 07:14:28.151304 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260924 07:14:28.151895 16785 heartbeater.cc:347] Connected to a master server at 127.10.130.254:42495
I20260924 07:14:28.151993 16785 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:28.152153 16785 heartbeater.cc:511] Master 127.10.130.254:42495 requested a full tablet report, sending...
I20260924 07:14:28.152486 16326 ts_manager.cc:194] Registered new tserver with Master: 1da819ab6c5443678a1ebd1a6580c293 (127.10.130.195:37061)
I20260924 07:14:28.152805 16326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:33359
I20260924 07:14:28.155567 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/test-admin.keytab.
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:28.167518 10763 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260924 07:14:28.169013 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:28.168301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.10.130.193:41725)
0924 07:14:28.168435 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:28.168439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:28.168449 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:14:28.168475 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:28.168476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:28.168489 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:28.168554 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:28.168952 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58344: BlockingRecv error: recv got EOF from 127.0.0.1:58344 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":62,"threads_started":1}
W20260924 07:14:28.169185 10763 init.cc:244] Error while opening credential cache '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/krb5cc)
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:28.170121 10763 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:28.170967 10763 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.193@KRBTEST.COM
I20260924 07:14:28.178217 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:28.171520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58350 (local address 127.10.130.193:41725)
0924 07:14:28.171574 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:28.171576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:28.171599 (+ 23us) server_negotiation.cc:407] Connection header received
0924 07:14:28.171622 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:28.171622 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:28.171633 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:28.171674 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:28.172054 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.172786 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.173459 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.173768 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.173826 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:28.175775 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:28.175785 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:28.175785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:28.175792 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:28.177203 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:28.177508 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:28.177511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:28.177512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:28.177559 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:28.177718 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:28.177718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:28.177719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:28.177844 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:28.177965 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:28.178040 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 07:14:28.178120 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260924 07:14:28.179757 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:28.179322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58366 (local address 127.10.130.193:41725)
0924 07:14:28.179357 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:28.179358 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:28.179389 (+ 31us) server_negotiation.cc:407] Connection header received
0924 07:14:28.179433 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:28.179434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:28.179443 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:28.179479 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:28.179715 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58366: BlockingRecv error: recv got EOF from 127.0.0.1:58366 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 07:14:28.179836 10763 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/krb5cc' of type FILE
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:28.180693 10763 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:28.181494 10763 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.193@KRBTEST.COM
I20260924 07:14:28.187116 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:28.181979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58378 (local address 127.10.130.193:41725)
0924 07:14:28.182021 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:28.182023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:28.182031 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:28.182091 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:28.182092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:28.182098 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:28.182148 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:28.182437 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.182909 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.183500 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.183709 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.183751 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:28.185446 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:28.185447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:28.185447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:28.185452 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:28.186407 (+ 955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:28.186659 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:28.186659 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:28.186660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:28.186697 (+ 37us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:28.186807 (+ 110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:28.186808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:28.186808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:28.186877 (+ 69us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:28.186935 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:28.186995 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 07:14:28.187043 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260924 07:14:28.188294 16792 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:28.187905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.10.130.193:41725)
0924 07:14:28.187937 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:28.187938 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:28.187949 (+ 11us) server_negotiation.cc:407] Connection header received
0924 07:14:28.188017 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:28.188018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:28.188026 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:28.188064 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:28.188256 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58380: BlockingRecv error: recv got EOF from 127.0.0.1:58380 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260924 07:14:28.188370 10763 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790234044967003-10763-0/krb5kdc/krb5cc' of type FILE
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:28.189110 10763 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:28.189776 10763 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16281](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.193@KRBTEST.COM
I20260924 07:14:28.195381 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:28.190178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58382 (local address 127.10.130.193:41725)
0924 07:14:28.190228 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:28.190230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:28.190266 (+ 36us) server_negotiation.cc:407] Connection header received
0924 07:14:28.190308 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:28.190309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:28.190317 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:28.190359 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:28.190647 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.191078 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.191685 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:28.191866 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:28.191917 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:28.193575 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:28.193576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:28.193577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:28.193583 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:28.194520 (+ 937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:28.194787 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:28.194788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:28.194788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:28.194835 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:28.194984 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:28.194985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:28.194986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:28.195066 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:28.195137 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:28.195232 (+ 95us) server_negotiation.cc:299] Negotiation successful
0924 07:14:28.195302 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 07:14:28.195961 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 16388
I20260924 07:14:28.200536 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 16522
I20260924 07:14:28.204335 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 16656
I20260924 07:14:28.207885 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 16294
2026-09-24T07:14:28Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1727 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16805](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16805](info): set up 2 sockets
Sep 24 07:14:28 dist-test-slave-7q7m krb5kdc[16805](info): commencing operation
krb5kdc: starting...
W20260924 07:14:29.159786 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:29 dist-test-slave-7q7m krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:29Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.254.keytab.
Entry for principal oryx/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.254.keytab.
I20260924 07:14:29.259779 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:40779
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.254.keytab
--principal=oryx/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:40779
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:29.336028 16818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:29.336220 16818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:29.336239 16818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:29.337352 16818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:29.337391 16818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:29.337404 16818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:29.337422 16818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:29.337447 16818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:29.338856 16818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:40779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.254.keytab
--principal=oryx/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:40779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:29.339064 16818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:29.339260 16818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:29.343575 16826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:29.343596 16823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:29.343569 16824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:29.343761 16818 server_base.cc:1034] running on GCE node
I20260924 07:14:29.344113 16818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:29.344370 16818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:29.345501 16818 hybrid_clock.cc:648] HybridClock initialized: now 1790234069345486 us; error 31 us; skew 500 ppm
Sep 24 07:14:29 dist-test-slave-7q7m krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:29.347734 16818 init.cc:377] Logged in from keytab as oryx/127.10.130.254@KRBTEST.COM (short username oryx)
I20260924 07:14:29.348163 16818 webserver.cc:460] Webserver started at http://127.10.130.254:43389/ using document root <none> and password file <none>
I20260924 07:14:29.348357 16818 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:29.348398 16818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:29.348512 16818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:29.349475 16818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "433bb3767a8c44619896a663d6fcdf67"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "b89670973c86cdbca2f19fab3b600922"
server_key_iv: "3c4961e70f2ff2b45aaef0a732033522"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.349787 16818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "433bb3767a8c44619896a663d6fcdf67"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "b89670973c86cdbca2f19fab3b600922"
server_key_iv: "3c4961e70f2ff2b45aaef0a732033522"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.350900 16818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 07:14:29.351476 16833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:29.351662 16818 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:29.351768 16818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "433bb3767a8c44619896a663d6fcdf67"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "b89670973c86cdbca2f19fab3b600922"
server_key_iv: "3c4961e70f2ff2b45aaef0a732033522"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.351835 16818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:29.368165 16818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:29.369704 16818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:29.369827 16818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:29.373404 16818 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:40779
I20260924 07:14:29.373441 16885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:40779 every 8 connection(s)
I20260924 07:14:29.373739 16818 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:29.374580 16886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:29.376714 16886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67: Bootstrap starting.
I20260924 07:14:29.377234 16886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:29.377478 16886 log.cc:824] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:29.378083 16886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67: No bootstrap required, opened a new log
I20260924 07:14:29.379551 16886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } }
I20260924 07:14:29.379678 16886 raft_consensus.cc:374] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:29.379699 16886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 433bb3767a8c44619896a663d6fcdf67, State: Initialized, Role: FOLLOWER
I20260924 07:14:29.379842 16886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } }
I20260924 07:14:29.379902 16886 raft_consensus.cc:388] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:29.379921 16886 raft_consensus.cc:482] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:29.379966 16886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:29.380682 16886 raft_consensus.cc:504] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } }
I20260924 07:14:29.380767 16886 leader_election.cc:302] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 433bb3767a8c44619896a663d6fcdf67; no voters:
I20260924 07:14:29.380913 16886 leader_election.cc:288] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:29.380950 16889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:29.381078 16889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [term 1 LEADER]: Becoming Leader. State: Replica: 433bb3767a8c44619896a663d6fcdf67, State: Running, Role: LEADER
I20260924 07:14:29.381135 16886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:29.381156 16889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } }
I20260924 07:14:29.381500 16890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } }
I20260924 07:14:29.381515 16891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 433bb3767a8c44619896a663d6fcdf67. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } }
I20260924 07:14:29.381654 16890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:29.381659 16891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:29.382030 16898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:29.382467 16898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:29.383271 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 16818
I20260924 07:14:29.383460 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:29.383785 10763 external_mini_cluster.cc:1428] Setting key 92bc5abd16ace79688dbb581114a2308
I20260924 07:14:29.383841 16898 catalog_manager.cc:1347] Generated new cluster ID: e06d2fbb994c47c180c5145efd658cc4
I20260924 07:14:29.383878 16898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:14:29 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
I20260924 07:14:29.391628 16898 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:29.392308 16898 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:29.393306 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:29.384408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35910 (local address 127.10.130.254:40779)
0924 07:14:29.384901 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:29.384911 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:29.385235 (+ 324us) server_negotiation.cc:407] Connection header received
0924 07:14:29.385445 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:29.385453 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:29.385522 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:29.385632 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:29.386307 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.387163 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.387877 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.388330 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.388430 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:29.390408 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:29.390412 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:29.390413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:29.390423 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:29.391956 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:29.392269 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:29.392272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:29.392273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:29.392349 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:29.392530 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:29.392531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:29.392531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:29.392733 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:29.392866 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:29.392972 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 07:14:29.393103 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":70,"threads_started":1}
I20260924 07:14:29.396301 16898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67: Generated new TSK 0
I20260924 07:14:29.396438 16898 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab.
Entry for principal oryx/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab.
I20260924 07:14:29.423785 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:40779
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab
--principal=oryx/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:29.499646 16912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:29.499847 16912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:29.499869 16912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:29.500983 16912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:29.501024 16912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:29.501049 16912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:29.502388 16912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab
--principal=oryx/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:40779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:29.502597 16912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:29.502806 16912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:29.507387 16918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:29.507313 16920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:29.507319 16917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:29.507344 16912 server_base.cc:1034] running on GCE node
I20260924 07:14:29.507717 16912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:29.507921 16912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:29.509049 16912 hybrid_clock.cc:648] HybridClock initialized: now 1790234069509048 us; error 33 us; skew 500 ppm
Sep 24 07:14:29 dist-test-slave-7q7m krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:29.511202 16912 init.cc:377] Logged in from keytab as oryx/127.10.130.193@KRBTEST.COM (short username oryx)
I20260924 07:14:29.511579 16912 webserver.cc:460] Webserver started at http://127.10.130.193:33719/ using document root <none> and password file <none>
I20260924 07:14:29.511757 16912 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:29.511802 16912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:29.511921 16912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:29.512878 16912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "779bea9f583b47c09308fb3a22d42447"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "cb9114d8c7148266aeefa9594897d9f9"
server_key_iv: "c25a3281f15690f94c79f8db219c2461"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.513185 16912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "779bea9f583b47c09308fb3a22d42447"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "cb9114d8c7148266aeefa9594897d9f9"
server_key_iv: "c25a3281f15690f94c79f8db219c2461"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.514338 16912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 07:14:29.514922 16927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:29.515058 16912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:29.515132 16912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "779bea9f583b47c09308fb3a22d42447"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "cb9114d8c7148266aeefa9594897d9f9"
server_key_iv: "c25a3281f15690f94c79f8db219c2461"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.515192 16912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:29.545998 16912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:29.547559 16912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:29.547679 16912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:29.547869 16912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:29.548162 16912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:29.548195 16912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:29.548223 16912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:29.548245 16912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:29.553561 16912 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:41189
I20260924 07:14:29.553596 17040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:41189 every 8 connection(s)
I20260924 07:14:29.553874 16912 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:14:29.557622 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 16912
I20260924 07:14:29.557706 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:29.557906 10763 external_mini_cluster.cc:1428] Setting key e1bb3ef2ed3ea84c84c5837362bdf3d3
Sep 24 07:14:29 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
I20260924 07:14:29.562908 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:29.554618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:57063 (local address 127.10.130.254:40779)
0924 07:14:29.554677 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:29.554679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:29.554849 (+ 170us) server_negotiation.cc:407] Connection header received
0924 07:14:29.555319 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:29.555320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:29.555331 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:29.555384 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:29.556285 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.556775 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.557844 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.558117 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.558178 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:29.560282 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:29.560283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:29.560284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:29.560291 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:29.561518 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:29.561979 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:29.561979 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:29.561980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:29.562025 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:29.562279 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:29.562280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:29.562281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:29.562425 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:29.562495 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:29.562735 (+ 240us) server_negotiation.cc:299] Negotiation successful
0924 07:14:29.562816 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 07:14:29.562963 17043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:29.554582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.193:57063)
0924 07:14:29.554737 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:29.554772 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:29.555243 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:29.555452 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:29.555456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:29.556028 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:29.556214 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:29.556219 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.556876 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.556877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:29.557734 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:29.557737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.558197 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.558198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:29.558332 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:29.558777 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:29.558786 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:29.560183 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:29.561592 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:29.561599 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:29.561601 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:29.561904 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:29.562082 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:29.562083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:29.562084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:29.562210 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:29.562490 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:29.562492 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:29.562596 (+ 104us) client_negotiation.cc:769] Sending connection context
0924 07:14:29.562698 (+ 102us) client_negotiation.cc:240] Negotiation successful
0924 07:14:29.562814 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260924 07:14:29.563410 17041 heartbeater.cc:347] Connected to a master server at 127.10.130.254:40779
I20260924 07:14:29.563490 17041 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:29.563653 17041 heartbeater.cc:511] Master 127.10.130.254:40779 requested a full tablet report, sending...
I20260924 07:14:29.564222 16850 ts_manager.cc:194] Registered new tserver with Master: 779bea9f583b47c09308fb3a22d42447 (127.10.130.193:41189)
I20260924 07:14:29.565093 16850 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:57063
WARNING: no policy specified for oryx/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab.
Entry for principal oryx/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab.
I20260924 07:14:29.580348 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:40779
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab
--principal=oryx/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:29.656623 17046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:29.656811 17046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:29.656834 17046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:29.657910 17046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:29.657949 17046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:29.657975 17046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:29.659334 17046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab
--principal=oryx/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:40779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:29.659507 17046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:29.659689 17046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:29.664175 17051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:29.664180 17054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:29.664340 17046 server_base.cc:1034] running on GCE node
W20260924 07:14:29.664239 17052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:29.664530 17046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:29.664747 17046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:29.665872 17046 hybrid_clock.cc:648] HybridClock initialized: now 1790234069665865 us; error 28 us; skew 500 ppm
Sep 24 07:14:29 dist-test-slave-7q7m krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:29.668288 17046 init.cc:377] Logged in from keytab as oryx/127.10.130.194@KRBTEST.COM (short username oryx)
I20260924 07:14:29.668733 17046 webserver.cc:460] Webserver started at http://127.10.130.194:39847/ using document root <none> and password file <none>
I20260924 07:14:29.668943 17046 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:29.668993 17046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:29.669121 17046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:29.670077 17046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "bdc83d87f36a4efeb4076c1e3c68917e"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "abab960cfdaee016a769b1e9b4b40782"
server_key_iv: "af7ef700cc04d2ad10e4a7e25de2b955"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.670405 17046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "bdc83d87f36a4efeb4076c1e3c68917e"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "abab960cfdaee016a769b1e9b4b40782"
server_key_iv: "af7ef700cc04d2ad10e4a7e25de2b955"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.671597 17046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:14:29.672276 17061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:29.672426 17046 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:29.672506 17046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "bdc83d87f36a4efeb4076c1e3c68917e"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "abab960cfdaee016a769b1e9b4b40782"
server_key_iv: "af7ef700cc04d2ad10e4a7e25de2b955"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.672575 17046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:29.689824 17046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:29.691373 17046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:29.691507 17046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:29.691706 17046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:29.692009 17046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:29.692052 17046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:29.692083 17046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:29.692111 17046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:29.697938 17046 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:38631
I20260924 07:14:29.697973 17174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:38631 every 8 connection(s)
I20260924 07:14:29.698257 17046 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
Sep 24 07:14:29 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
I20260924 07:14:29.704147 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 17046
I20260924 07:14:29.704233 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:29.704437 10763 external_mini_cluster.cc:1428] Setting key 8181bc26d784ca3c8d439bc39e9e2da8
I20260924 07:14:29.706864 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:29.698898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:51779 (local address 127.10.130.254:40779)
0924 07:14:29.698955 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:29.698957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:29.699134 (+ 177us) server_negotiation.cc:407] Connection header received
0924 07:14:29.699580 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:29.699581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:29.699591 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:29.699666 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:29.700540 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.701036 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.701939 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.702121 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.702170 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:29.704065 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:29.704067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:29.704067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:29.704075 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:29.705684 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:29.706022 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:29.706023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:29.706023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:29.706082 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:29.706279 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:29.706280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:29.706280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:29.706409 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:29.706471 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:29.706748 (+ 277us) server_negotiation.cc:299] Negotiation successful
0924 07:14:29.706803 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 07:14:29.707074 17177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:29.698863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.194:51779)
0924 07:14:29.699015 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:29.699053 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:29.699491 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:29.699722 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:29.699725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:29.700289 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:29.700464 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:29.700469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.701104 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.701105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:29.701879 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:29.701881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.702174 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.702174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:29.702264 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:29.702691 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:29.702699 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:29.703955 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:29.705758 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:29.705763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:29.705765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:29.705960 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:29.706147 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:29.706148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:29.706148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:29.706231 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:29.706469 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:29.706473 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:29.706622 (+ 149us) client_negotiation.cc:769] Sending connection context
0924 07:14:29.706722 (+ 100us) client_negotiation.cc:240] Negotiation successful
0924 07:14:29.706871 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":71,"threads_started":1}
I20260924 07:14:29.707470 17175 heartbeater.cc:347] Connected to a master server at 127.10.130.254:40779
I20260924 07:14:29.707556 17175 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:29.707710 17175 heartbeater.cc:511] Master 127.10.130.254:40779 requested a full tablet report, sending...
I20260924 07:14:29.708048 16850 ts_manager.cc:194] Registered new tserver with Master: bdc83d87f36a4efeb4076c1e3c68917e (127.10.130.194:38631)
I20260924 07:14:29.708400 16850 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:51779
WARNING: no policy specified for oryx/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab.
Entry for principal oryx/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab.
I20260924 07:14:29.726238 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:40779
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab
--principal=oryx/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:29.801781 17180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:29.801954 17180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:29.801973 17180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:29.803027 17180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:29.803059 17180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:29.803083 17180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:29.804457 17180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab
--principal=oryx/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:40779
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:29.804618 17180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:29.804780 17180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:29.809163 17188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:29.809257 17186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:29.809165 17185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:29.809393 17180 server_base.cc:1034] running on GCE node
I20260924 07:14:29.809566 17180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:29.809818 17180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:29.810976 17180 hybrid_clock.cc:648] HybridClock initialized: now 1790234069810955 us; error 36 us; skew 500 ppm
Sep 24 07:14:29 dist-test-slave-7q7m krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:29.813180 17180 init.cc:377] Logged in from keytab as oryx/127.10.130.195@KRBTEST.COM (short username oryx)
I20260924 07:14:29.813551 17180 webserver.cc:460] Webserver started at http://127.10.130.195:42319/ using document root <none> and password file <none>
I20260924 07:14:29.813726 17180 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:29.813776 17180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:29.813896 17180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:29.814819 17180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "e25329d24dfa425f8984014b7813fea8"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "0c89d368f04caef54a6ca9b06509973d"
server_key_iv: "f1b6e507a4926a78f13c324a3235fc65"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.815117 17180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "e25329d24dfa425f8984014b7813fea8"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "0c89d368f04caef54a6ca9b06509973d"
server_key_iv: "f1b6e507a4926a78f13c324a3235fc65"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.816254 17180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:14:29.816860 17195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:29.816994 17180 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:29.817070 17180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "e25329d24dfa425f8984014b7813fea8"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "0c89d368f04caef54a6ca9b06509973d"
server_key_iv: "f1b6e507a4926a78f13c324a3235fc65"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.817124 17180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:29.831864 17180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:29.833493 17180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:29.833613 17180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:29.833808 17180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:29.834060 17180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:29.834091 17180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:29.834117 17180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:29.834139 17180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:29.839437 17180 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:38491
I20260924 07:14:29.839470 17308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:38491 every 8 connection(s)
I20260924 07:14:29.839730 17180 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
Sep 24 07:14:29 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
I20260924 07:14:29.847340 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:29.840342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:47585 (local address 127.10.130.254:40779)
0924 07:14:29.840386 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:29.840388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:29.840551 (+ 163us) server_negotiation.cc:407] Connection header received
0924 07:14:29.841014 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:29.841016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:29.841025 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:29.841072 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:29.841886 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.842362 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.843236 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.843419 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.843466 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:29.845206 (+ 1740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:29.845208 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:29.845208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:29.845215 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:29.846364 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:29.846651 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:29.846651 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:29.846651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:29.846722 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:29.846897 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:29.846898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:29.846898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:29.846986 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:29.847028 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:29.847217 (+ 189us) server_negotiation.cc:299] Negotiation successful
0924 07:14:29.847276 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:14:29.847496 17311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:29.840314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.195:47585)
0924 07:14:29.840450 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:29.840480 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:29.840936 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:29.841125 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:29.841130 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:29.841641 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:29.841821 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:29.841826 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.842418 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.842419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:29.843171 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:29.843173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.843474 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.843475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:29.843562 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:29.843913 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:29.843920 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:29.845129 (+ 1209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:29.846421 (+ 1292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:29.846423 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:29.846425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:29.846602 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:29.846765 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:29.846766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:29.846766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:29.846856 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:29.847025 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:29.847027 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:29.847124 (+ 97us) client_negotiation.cc:769] Sending connection context
0924 07:14:29.847194 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 07:14:29.847300 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":79,"threads_started":1}
I20260924 07:14:29.847925 17309 heartbeater.cc:347] Connected to a master server at 127.10.130.254:40779
I20260924 07:14:29.848034 17309 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:29.848199 17309 heartbeater.cc:511] Master 127.10.130.254:40779 requested a full tablet report, sending...
I20260924 07:14:29.848510 16850 ts_manager.cc:194] Registered new tserver with Master: e25329d24dfa425f8984014b7813fea8 (127.10.130.195:38491)
I20260924 07:14:29.848848 16850 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:47585
I20260924 07:14:29.849952 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 17180
I20260924 07:14:29.850032 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:29.850201 10763 external_mini_cluster.cc:1428] Setting key 26a3f942da6684df6046839a4f23bd17
I20260924 07:14:29.850943 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.130.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.130.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.130.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.253.keytab.
Entry for principal oryx/127.10.130.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.253.keytab.
I20260924 07:14:29.872879 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.253:34245
--webserver_interface=127.10.130.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.253.keytab
--principal=oryx/127.10.130.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:40779,127.10.130.253:34245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:29.950281 17314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:29.950585 17314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:29.950645 17314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:29.952328 17314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:29.952381 17314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:29.952400 17314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:29.952415 17314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:29.952432 17314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:29.954123 17314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:40779,127.10.130.253:34245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.253.keytab
--principal=oryx/127.10.130.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.253:34245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.130.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
W20260924 07:14:29.954224 17314 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.10.130.254:40779,127.10.130.253:34245'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260924 07:14:29.954365 17314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:29.954574 17314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:29.958884 17319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:29.958936 17320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:29.958940 17322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:29.959082 17314 server_base.cc:1034] running on GCE node
I20260924 07:14:29.959427 17314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:29.959692 17314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:29.960819 17314 hybrid_clock.cc:648] HybridClock initialized: now 1790234069960808 us; error 31 us; skew 500 ppm
Sep 24 07:14:29 dist-test-slave-7q7m krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:29.963034 17314 init.cc:377] Logged in from keytab as oryx/127.10.130.253@KRBTEST.COM (short username oryx)
I20260924 07:14:29.963383 17314 webserver.cc:460] Webserver started at http://127.10.130.253:33743/ using document root <none> and password file <none>
I20260924 07:14:29.963538 17314 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:29.963574 17314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:29.963662 17314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:29.964613 17314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/data/instance:
uuid: "5b151bdf2b7f4959b1d7bcb788a3c687"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "63ea69c6610ff443cf928596dd366f37"
server_key_iv: "5a6bd1bf28f0b933667d7c700af6634f"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.964888 17314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/wal/instance:
uuid: "5b151bdf2b7f4959b1d7bcb788a3c687"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "63ea69c6610ff443cf928596dd366f37"
server_key_iv: "5a6bd1bf28f0b933667d7c700af6634f"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.965979 17314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:29.966604 17329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:29.966748 17314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:29.966823 17314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/wal
uuid: "5b151bdf2b7f4959b1d7bcb788a3c687"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "63ea69c6610ff443cf928596dd366f37"
server_key_iv: "5a6bd1bf28f0b933667d7c700af6634f"
server_key_version: "encryptionkey@0"
I20260924 07:14:29.966873 17314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:29.978766 17314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:29.980480 17314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:29.980581 17314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:29.984237 17314 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.253:34245
I20260924 07:14:29.984288 17381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.253:34245 every 8 connection(s)
I20260924 07:14:29.984592 17314 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/data/info.pb
I20260924 07:14:29.985430 17382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:29.986464 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 17314
I20260924 07:14:29.987038 17382 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 07:14:29 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.253@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
Sep 24 07:14:29 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.130.253@KRBTEST.COM
I20260924 07:14:29.994706 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:29.987660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35924 (local address 127.10.130.254:40779)
0924 07:14:29.987704 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:29.987706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:29.987846 (+ 140us) server_negotiation.cc:407] Connection header received
0924 07:14:29.987896 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:29.987897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:29.987907 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:29.987963 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:29.988483 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.988965 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.989934 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.990185 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.990231 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:29.992311 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:29.992313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:29.992313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:29.992321 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:29.993465 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:29.993857 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:29.993858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:29.993859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:29.993916 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:29.994115 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:29.994116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:29.994116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:29.994255 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:29.994312 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:29.994546 (+ 234us) server_negotiation.cc:299] Negotiation successful
0924 07:14:29.994620 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 07:14:29.994874 17386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:29.987587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:35924)
0924 07:14:29.987731 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:29.987765 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:29.987846 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:29.988007 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:29.988009 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:29.988242 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:29.988393 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:29.988396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.989047 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.989048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:29.989861 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:29.989863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.990243 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.990244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:29.990353 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:29.990820 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:29.990828 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:29.992231 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:29.993518 (+ 1287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:29.993523 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:29.993525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:29.993788 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:29.993969 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:29.993969 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:29.993970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:29.994060 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:29.994320 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:29.994325 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:29.994436 (+ 111us) client_negotiation.cc:769] Sending connection context
0924 07:14:29.994519 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 07:14:29.994649 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260924 07:14:29.995290 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:29.987030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40620 (local address 127.10.130.253:34245)
0924 07:14:29.987175 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:29.987183 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:29.987206 (+ 23us) server_negotiation.cc:407] Connection header received
0924 07:14:29.987669 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:29.987678 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:29.987743 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:29.987873 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:29.988446 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.989373 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.990084 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.990424 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.990490 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:29.992802 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:29.992808 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:29.992808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:29.992814 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:29.994189 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:29.994485 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:29.994488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:29.994488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:29.994536 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:29.994695 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:29.994695 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:29.994696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:29.994860 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:29.994989 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:29.995121 (+ 132us) server_negotiation.cc:299] Negotiation successful
0924 07:14:29.995202 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260924 07:14:29.995515 17382 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 07:14:29 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.253@KRBTEST.COM for oryx/127.10.130.253@KRBTEST.COM
I20260924 07:14:30.000372 17386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:29.995735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:34245 (local address 127.0.0.1:40634)
0924 07:14:29.995797 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:29.995805 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:29.995836 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:29.995921 (+ 85us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:29.995922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:29.996059 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:29.996086 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:29.996087 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.996597 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.996598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:29.997045 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:29.997047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.997308 (+ 261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.997309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:29.997350 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:29.997610 (+ 260us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:29.997613 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:29.998572 (+ 959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:29.999645 (+ 1073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:29.999646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:29.999646 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:29.999783 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:29.999917 (+ 134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:29.999917 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:29.999917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:29.999957 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.000162 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.000163 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.000193 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.000235 (+ 42us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.000298 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260924 07:14:30.000427 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:29.995760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40634 (local address 127.10.130.253:34245)
0924 07:14:29.995837 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:29.995838 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:29.995845 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:29.995874 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:29.995874 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:29.995881 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:29.995917 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:29.996127 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.996546 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.997094 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:29.997265 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:29.997305 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:29.998644 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:29.998645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:29.998645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:29.998650 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:29.999576 (+ 926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:29.999832 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:29.999833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:29.999833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:29.999872 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.000032 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.000033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.000033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.000118 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.000161 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.000252 (+ 91us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.000337 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":61}
I20260924 07:14:30.001474 17382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: Bootstrap starting.
I20260924 07:14:30.002200 17382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:30.002431 17382 log.cc:824] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:30.003021 17382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: No bootstrap required, opened a new log
I20260924 07:14:30.004354 17382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } }
I20260924 07:14:30.004468 17382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:30.004499 17382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b151bdf2b7f4959b1d7bcb788a3c687, State: Initialized, Role: FOLLOWER
I20260924 07:14:30.004606 17382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } }
I20260924 07:14:30.004810 17388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } } }
I20260924 07:14:30.004858 17382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:30.004894 17388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 07:14:30.006212 17399 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:14:30.006258 17399 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:14:30.007619 17314 master_runner.cc:428] Detected that this master 5b151bdf2b7f4959b1d7bcb788a3c687 is joining an existing cluster
I20260924 07:14:30.007665 17314 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.130.253:34245
I20260924 07:14:30.014159 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.008763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40642 (local address 127.10.130.253:34245)
0924 07:14:30.008791 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.008793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.008834 (+ 41us) server_negotiation.cc:407] Connection header received
0924 07:14:30.008883 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.008883 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.008891 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.008934 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.009240 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.009674 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.010684 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.010930 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.010986 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.011916 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.011918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.011918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.011923 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.013194 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.013461 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.013461 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.013462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.013507 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.013689 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.013690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.013690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.013816 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.013850 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.014032 (+ 182us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.014100 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:14:30.014379 17405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.008598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:34245 (local address 127.0.0.1:40642)
0924 07:14:30.008788 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.008797 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.008846 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.008941 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.008941 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.009111 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.009181 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.009183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.009735 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.009735 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.010605 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.010608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.011007 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.011008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.011072 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.011450 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.011454 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.011840 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.013262 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.013263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.013264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.013408 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.013564 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.013564 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.013564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.013620 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.013896 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.013897 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.013940 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.014215 (+ 275us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.014295 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":4,"spinlock_wait_cycles":9600,"thread_start_us":110,"threads_started":1}
I20260924 07:14:30.014593 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.008639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35932 (local address 127.10.130.254:40779)
0924 07:14:30.008679 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.008681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.008818 (+ 137us) server_negotiation.cc:407] Connection header received
0924 07:14:30.008906 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.008907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.008916 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.008943 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.009665 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.010121 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.010819 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.011070 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.011098 (+ 28us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.011961 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.011963 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.011963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.011970 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.013480 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.013821 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.013822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.013822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.013880 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.014053 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.014054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.014054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.014175 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.014236 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.014426 (+ 190us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.014505 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:14:30.014999 17406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.008597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:35932)
0924 07:14:30.008757 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.008766 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.008840 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.008997 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.008998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.009531 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.009579 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.009581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.010186 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.010187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.010734 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.010735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.011159 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.011159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.011230 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.011568 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.011572 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.011856 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.013554 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.013554 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.013555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.013757 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.013943 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.013944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.013944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.013998 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.014263 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.014264 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.014314 (+ 50us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.014694 (+ 380us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.014785 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.254@KRBTEST.COM for oryx/127.10.130.253@KRBTEST.COM
I20260924 07:14:30.023211 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.016795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40652 (local address 127.10.130.253:34245)
0924 07:14:30.016830 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.016831 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.016939 (+ 108us) server_negotiation.cc:407] Connection header received
0924 07:14:30.016989 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.016990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.016998 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.017058 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.017440 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.018115 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.018801 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.019089 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.019152 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.021110 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.021111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.021111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.021116 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.022223 (+ 1107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.022625 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.022625 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.022625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.022677 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.022840 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.022841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.022841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.022917 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.022956 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.023111 (+ 155us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.023159 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:14:30.023324 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.016743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:34245 (local address 127.0.0.1:40652)
0924 07:14:30.016843 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.016882 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.016942 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.017063 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.017065 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.017335 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.017377 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.017378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.018178 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.018179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.018735 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.018737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.019152 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.019152 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.019199 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.019595 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.019598 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.021037 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.022292 (+ 1255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.022309 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.022311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.022558 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.022726 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.022727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.022727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.022792 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.022964 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.022970 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.023018 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.023096 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.023206 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":59,"threads_started":1}
I20260924 07:14:30.023818 16850 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.130.253:34245: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687"
member_type: NON_VOTER
last_known_addr {
host: "127.10.130.253"
port: 34245
}
attrs {
promote: true
}
}
dest_uuid: "433bb3767a8c44619896a663d6fcdf67"
cas_config_opid_index: -1
I20260924 07:14:30.024106 16850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true } }
I20260924 07:14:30.024528 17409 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5b151bdf2b7f4959b1d7bcb788a3c687 (127.10.130.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true } } }
I20260924 07:14:30.024755 17356 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:30.025527 17409 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.130.253:34245
I20260924 07:14:30.025527 17356 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 FOLLOWER]: Refusing update from remote peer 433bb3767a8c44619896a663d6fcdf67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260924 07:14:30.025727 17408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true } } }
I20260924 07:14:30.025839 17314 master.cc:537] Master@127.10.130.253:34245 shutting down...
I20260924 07:14:30.026315 17408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:30.025880 17409 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 07:14:30.027845 16834 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260924 07:14:30.028299 16834 consensus_peers.cc:487] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 -> Peer 5b151bdf2b7f4959b1d7bcb788a3c687 (127.10.130.253:34245): Couldn't send request to peer 5b151bdf2b7f4959b1d7bcb788a3c687. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 07:14:30.028445 17314 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:30.028532 17314 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:30.028577 17314 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: stopping tablet replica
I20260924 07:14:30.040661 17314 master.cc:552] Master@127.10.130.253:34245 shutdown complete.
I20260924 07:14:30.040702 17314 master_runner.cc:305] Clearing existing system tablet
I20260924 07:14:30.040807 17314 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:30.040841 17314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:30.041344 17314 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:30.041870 17413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.041985 17314 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:30.042033 17314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.042054 17314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/wal
uuid: "5b151bdf2b7f4959b1d7bcb788a3c687"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "63ea69c6610ff443cf928596dd366f37"
server_key_iv: "5a6bd1bf28f0b933667d7c700af6634f"
server_key_version: "encryptionkey@0"
I20260924 07:14:30.042591 17314 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:30.043782 17314 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 07:14:30.043838 17314 log.cc:1196] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260924 07:14:30.044023 17314 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: Deleting consensus metadata
I20260924 07:14:30.044129 17314 master_runner.cc:315] Copying system tablet from 127.10.130.254:40779
I20260924 07:14:30.044636 17314 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: tablet copy: Beginning tablet copy session from remote peer at address 127.10.130.254:40779
I20260924 07:14:30.049872 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.044935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35946 (local address 127.10.130.254:40779)
0924 07:14:30.044973 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.044975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.045016 (+ 41us) server_negotiation.cc:407] Connection header received
0924 07:14:30.045079 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.045080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.045092 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.045134 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.045445 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.045946 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.046692 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.046874 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.046925 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.047761 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.047762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.047763 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.047769 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.048955 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.049243 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.049244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.049244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.049291 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.049447 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.049448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.049448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.049543 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.049595 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.049719 (+ 124us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.049781 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:14:30.049887 17419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.044865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:35946)
0924 07:14:30.044959 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.044969 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.045031 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.045146 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.045147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.045336 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.045386 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.045387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.046005 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.046006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.046617 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.046619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.046927 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.046928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.046991 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.047423 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.047427 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.047689 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.049023 (+ 1334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.049023 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.049024 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.049186 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.049340 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.049341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.049341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.049399 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.049592 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.049593 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.049633 (+ 40us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.049693 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.049783 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"thread_start_us":55,"threads_started":1}
I20260924 07:14:30.050212 16870 tablet_copy_service.cc:140] P 433bb3767a8c44619896a663d6fcdf67: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5b151bdf2b7f4959b1d7bcb788a3c687 ({username='oryx', principal='oryx/127.10.130.253@KRBTEST.COM'} at 127.0.0.1:35946)
I20260924 07:14:30.050280 16870 tablet_copy_service.cc:161] P 433bb3767a8c44619896a663d6fcdf67: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5b151bdf2b7f4959b1d7bcb788a3c687 at {username='oryx', principal='oryx/127.10.130.253@KRBTEST.COM'} at 127.0.0.1:35946: session id = 5b151bdf2b7f4959b1d7bcb788a3c687-00000000000000000000000000000000
I20260924 07:14:30.050832 16870 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 07:14:30.051308 17314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 07:14:30.052160 17314 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: tablet copy: Starting download of 0 data blocks...
I20260924 07:14:30.052266 17314 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: tablet copy: Starting download of 1 WAL segments...
I20260924 07:14:30.053035 17314 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 07:14:30.053737 16870 tablet_copy_service.cc:342] P 433bb3767a8c44619896a663d6fcdf67: Request end of tablet copy session 5b151bdf2b7f4959b1d7bcb788a3c687-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.130.253@KRBTEST.COM'} at 127.0.0.1:35946
I20260924 07:14:30.053787 16870 tablet_copy_service.cc:433] P 433bb3767a8c44619896a663d6fcdf67: ending tablet copy session 5b151bdf2b7f4959b1d7bcb788a3c687-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5b151bdf2b7f4959b1d7bcb788a3c687
I20260924 07:14:30.055884 17314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:30.057004 17420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:30.057029 17421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:30.057142 17314 server_base.cc:1034] running on GCE node
W20260924 07:14:30.057051 17423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:30.057353 17314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:30.057477 17314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:30.058581 17314 hybrid_clock.cc:648] HybridClock initialized: now 1790234070058573 us; error 29 us; skew 500 ppm
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:30.059840 17314 init.cc:377] Logged in from keytab as oryx/127.10.130.253@KRBTEST.COM (short username oryx)
I20260924 07:14:30.060103 17314 webserver.cc:460] Webserver started at http://127.10.130.253:39777/ using document root <none> and password file <none>
I20260924 07:14:30.060213 17314 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:30.060246 17314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:30.060866 17314 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:30.061383 17430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.061582 17314 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:30.061626 17314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/wal
uuid: "5b151bdf2b7f4959b1d7bcb788a3c687"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "63ea69c6610ff443cf928596dd366f37"
server_key_iv: "5a6bd1bf28f0b933667d7c700af6634f"
server_key_version: "encryptionkey@0"
I20260924 07:14:30.061685 17314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:30.065585 17314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:30.066771 17314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:30.069206 17314 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.253:34245
I20260924 07:14:30.069605 17314 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-1/data/info.pb
I20260924 07:14:30.069655 17482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.253:34245 every 8 connection(s)
I20260924 07:14:30.070227 17483 sys_catalog.cc:263] Verifying existing consensus state
I20260924 07:14:30.070555 17483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: Bootstrap starting.
I20260924 07:14:30.072526 17483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 07:14:30.072705 17483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: Bootstrap complete.
I20260924 07:14:30.072853 17483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true } }
I20260924 07:14:30.072914 17483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b151bdf2b7f4959b1d7bcb788a3c687, State: Initialized, Role: LEARNER
I20260924 07:14:30.072937 17483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true } }
I20260924 07:14:30.073074 17483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:30.073079 17486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true } } }
I20260924 07:14:30.073190 17486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: This master's current role is: LEARNER
I20260924 07:14:30.074282 17497 catalog_manager.cc:1259] Loaded cluster ID: e06d2fbb994c47c180c5145efd658cc4
I20260924 07:14:30.074352 17497 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: loading cluster ID for follower catalog manager: success
I20260924 07:14:30.075290 17497 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: acquiring CA information for follower catalog manager: success
I20260924 07:14:30.075564 17497 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 07:14:30.103070 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.096634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.10.130.253:34245)
0924 07:14:30.096764 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.096766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.096776 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:14:30.096799 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.096800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.096811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.096890 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.097390 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.098429 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.099103 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.099410 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.099482 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.100553 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.100555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.100555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.100562 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.101991 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.102277 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.102278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.102278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.102369 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.102505 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.102506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.102506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.102623 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.102713 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.102870 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.102949 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
WARNING: no policy specified for oryx/127.10.130.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.130.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.130.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.252.keytab.
Entry for principal oryx/127.10.130.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.252.keytab.
I20260924 07:14:30.124382 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.252:38911
--webserver_interface=127.10.130.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.252.keytab
--principal=oryx/127.10.130.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:40779,127.10.130.253:34245,127.10.130.252:38911
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:30.200392 17501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:30.200562 17501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:30.200580 17501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:30.201610 17501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:30.201642 17501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:30.201660 17501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:30.201674 17501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:30.201685 17501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:30.203028 17501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:40779,127.10.130.253:34245,127.10.130.252:38911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.252.keytab
--principal=oryx/127.10.130.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.252:38911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.130.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:30.203188 17501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:30.203337 17501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:30.207564 17507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:30.207602 17509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:30.207607 17501 server_base.cc:1034] running on GCE node
W20260924 07:14:30.207770 17506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:30.208014 17501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:30.208266 17501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:30.209395 17501 hybrid_clock.cc:648] HybridClock initialized: now 1790234070209384 us; error 27 us; skew 500 ppm
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:30.211536 17501 init.cc:377] Logged in from keytab as oryx/127.10.130.252@KRBTEST.COM (short username oryx)
I20260924 07:14:30.211889 17501 webserver.cc:460] Webserver started at http://127.10.130.252:43667/ using document root <none> and password file <none>
I20260924 07:14:30.212085 17501 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:30.212128 17501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:30.212213 17501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:30.213086 17501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/data/instance:
uuid: "a7e7537f3da642b095ac9def4634cb86"
format_stamp: "Formatted at 2026-09-24 07:14:30 on dist-test-slave-7q7m"
server_key: "d1eb00ab43abba3df1167ea8ca2d91d8"
server_key_iv: "b30321b40f6b67dfa7d0e33b6fc29e93"
server_key_version: "encryptionkey@0"
I20260924 07:14:30.213349 17501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/wal/instance:
uuid: "a7e7537f3da642b095ac9def4634cb86"
format_stamp: "Formatted at 2026-09-24 07:14:30 on dist-test-slave-7q7m"
server_key: "d1eb00ab43abba3df1167ea8ca2d91d8"
server_key_iv: "b30321b40f6b67dfa7d0e33b6fc29e93"
server_key_version: "encryptionkey@0"
I20260924 07:14:30.214401 17501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:30.214956 17516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.215103 17501 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:30.215171 17501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/wal
uuid: "a7e7537f3da642b095ac9def4634cb86"
format_stamp: "Formatted at 2026-09-24 07:14:30 on dist-test-slave-7q7m"
server_key: "d1eb00ab43abba3df1167ea8ca2d91d8"
server_key_iv: "b30321b40f6b67dfa7d0e33b6fc29e93"
server_key_version: "encryptionkey@0"
I20260924 07:14:30.215231 17501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:30.224881 17501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:30.226428 17501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:30.226538 17501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:30.229944 17501 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.252:38911
I20260924 07:14:30.229976 17568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.252:38911 every 8 connection(s)
I20260924 07:14:30.230249 17501 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/data/info.pb
I20260924 07:14:30.231046 17569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:30.232424 17569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.252@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
I20260924 07:14:30.238164 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 17501
I20260924 07:14:30.240515 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.233093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35960 (local address 127.10.130.254:40779)
0924 07:14:30.233161 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.233163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.233211 (+ 48us) server_negotiation.cc:407] Connection header received
0924 07:14:30.233679 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.233680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.233694 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.233744 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.234280 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.234753 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.235670 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.235862 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.235913 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.237866 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.237868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.237868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.237875 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.239214 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.239613 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.239614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.239614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.239683 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.239911 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.239911 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.239912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.240049 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.240098 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.240373 (+ 275us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.240433 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 07:14:30.240684 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.232963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:35960)
0924 07:14:30.233104 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.233146 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.233583 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.233814 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.233817 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.234026 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.234197 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.234201 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.234830 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.234831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.235590 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.235592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.235941 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.235942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.236074 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.236437 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.236448 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.237781 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.239275 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.239281 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.239282 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.239546 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.239737 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.239738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.239738 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.239859 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.240098 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.240102 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.240235 (+ 133us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.240342 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.240487 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260924 07:14:30.241170 17569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.130.252@KRBTEST.COM
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.252@KRBTEST.COM for oryx/127.10.130.253@KRBTEST.COM
I20260924 07:14:30.246284 17572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.238624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41894 (local address 127.10.130.252:38911)
0924 07:14:30.238765 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.238769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.238795 (+ 26us) server_negotiation.cc:407] Connection header received
0924 07:14:30.238836 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.238839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.238852 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.238921 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.239403 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.240396 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.241330 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.241730 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.241787 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.243874 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.243881 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.243881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.243887 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.245318 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.245598 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.245601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.245601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.245653 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.245804 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.245805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.245806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.245926 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.246039 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.246106 (+ 67us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.246194 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":76,"threads_started":1}
I20260924 07:14:30.246930 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.241364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:34245 (local address 127.0.0.1:40674)
0924 07:14:30.241403 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.241413 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.241446 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.241546 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.241547 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.241673 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.241715 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.241716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.242507 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.242508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.242978 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.242979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.243324 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.243325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.243386 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.243715 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.243719 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.244755 (+ 1036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.246103 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.246104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.246104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.246297 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.246451 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.246452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.246452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.246505 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.246703 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.246704 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.246741 (+ 37us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.246803 (+ 62us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.246878 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260924 07:14:30.246961 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.241429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40674 (local address 127.10.130.253:34245)
0924 07:14:30.241467 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.241469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.241476 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:30.241495 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.241495 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.241506 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.241544 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.241779 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.242410 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.243056 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.243261 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.243314 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.244832 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.244833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.244833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.244840 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.246031 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.246349 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.246350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.246351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.246402 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.246554 (+ 152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.246555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.246555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.246652 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.246704 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.246835 (+ 131us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.246903 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:14:30.247259 17569 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.252@KRBTEST.COM for oryx/127.10.130.252@KRBTEST.COM
I20260924 07:14:30.252084 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.247449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38911 (local address 127.0.0.1:41904)
0924 07:14:30.247478 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.247487 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.247524 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.247624 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.247625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.247732 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.247752 (+ 20us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.247753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.248341 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.248342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.248779 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.248780 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.249083 (+ 303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.249084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.249125 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.249391 (+ 266us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.249394 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.250322 (+ 928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.251368 (+ 1046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.251369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.251369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.251507 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.251634 (+ 127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.251634 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.251635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.251676 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.251821 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.251821 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.251851 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.251893 (+ 42us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.251953 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260924 07:14:30.252134 17572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.247477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41904 (local address 127.10.130.252:38911)
0924 07:14:30.247514 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.247515 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.247522 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:30.247564 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.247565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.247572 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.247620 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.247805 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.248283 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.248837 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.249034 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.249081 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.250394 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.250395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.250395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.250400 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.251311 (+ 911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.251556 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.251556 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.251557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.251593 (+ 36us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.251707 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.251707 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.251707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.251781 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.251819 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.251907 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.252025 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":24}
I20260924 07:14:30.253103 17569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: Bootstrap starting.
I20260924 07:14:30.253796 17569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:30.254024 17569 log.cc:824] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:30.254660 17569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: No bootstrap required, opened a new log
I20260924 07:14:30.255970 17569 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } }
I20260924 07:14:30.256115 17569 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:30.256145 17569 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7e7537f3da642b095ac9def4634cb86, State: Initialized, Role: FOLLOWER
I20260924 07:14:30.256250 17569 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } }
I20260924 07:14:30.256448 17574 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } } }
I20260924 07:14:30.256502 17569 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:30.256539 17574 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 07:14:30.257875 17585 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:14:30.257954 17585 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:14:30.259699 17501 master_runner.cc:428] Detected that this master a7e7537f3da642b095ac9def4634cb86 is joining an existing cluster
I20260924 07:14:30.259735 17501 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.130.252:38911
I20260924 07:14:30.267207 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.260633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35972 (local address 127.10.130.254:40779)
0924 07:14:30.260669 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.260671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.260779 (+ 108us) server_negotiation.cc:407] Connection header received
0924 07:14:30.260921 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.260922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.260931 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.260988 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.261315 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.262001 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.262981 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.263269 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.263303 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.264280 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.264282 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.264282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.264288 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.265619 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.266505 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.266506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.266506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.266556 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.266725 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.266725 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.266726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.266845 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.266903 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.267056 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.267116 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:14:30.267226 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.260614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:35972)
0924 07:14:30.260721 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.260730 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.260862 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.260993 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.260994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.261158 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.261225 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.261226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.262081 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.262082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.262899 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.262901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.263390 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.263391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.263449 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.263781 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.263785 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.264213 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.266144 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.266144 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.266145 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.266436 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.266608 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.266609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.266609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.266674 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.266928 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.266929 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.266963 (+ 34us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.267022 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.267127 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":1,"thread_start_us":65,"threads_started":1}
I20260924 07:14:30.267315 17572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.260810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41920 (local address 127.10.130.252:38911)
0924 07:14:30.260837 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.260839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.261144 (+ 305us) server_negotiation.cc:407] Connection header received
0924 07:14:30.261213 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.261214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.261220 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.261241 (+ 21us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.261772 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.262425 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.263390 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.263882 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.263923 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.264872 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.264873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.264874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.264879 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.266075 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.266502 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.266503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.266504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.266564 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.266764 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.266765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.266765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.266869 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.266926 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.267071 (+ 145us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.267232 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":107,"server-negotiator.queue_time_us":15}
I20260924 07:14:30.267494 17593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.260867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38911 (local address 127.0.0.1:41920)
0924 07:14:30.261102 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.261110 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.261159 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.261283 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.261283 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.261664 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.261706 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.261708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.262519 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.262519 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.263320 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.263323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.264017 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.264018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.264079 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.264454 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.264458 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.264785 (+ 327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.266140 (+ 1355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.266141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.266141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.266429 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.266639 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.266639 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.266640 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.266695 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.266927 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.266928 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.266970 (+ 42us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.267094 (+ 124us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.267409 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":60,"threads_started":1}
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234069, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.254@KRBTEST.COM for oryx/127.10.130.252@KRBTEST.COM
I20260924 07:14:30.290704 17572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.269159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41926 (local address 127.10.130.252:38911)
0924 07:14:30.269190 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.269192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.269217 (+ 25us) server_negotiation.cc:407] Connection header received
0924 07:14:30.269289 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.269290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.269296 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.269336 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.269601 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.270007 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.270930 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.271087 (+ 157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.271127 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.272824 (+ 1697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.272826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.272826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.272833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.289574 (+ 16741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.289944 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.289945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.289945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.290006 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.290192 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.290193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.290193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.290318 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.290377 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.290557 (+ 180us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.290620 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260924 07:14:30.290725 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.269123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38911 (local address 127.0.0.1:41926)
0924 07:14:30.269172 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.269182 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.269230 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.269349 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.269350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.269501 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.269540 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.269542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.270063 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.270064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.270866 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.270868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.271140 (+ 272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.271140 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.271208 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.271503 (+ 295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.271507 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.272725 (+ 1218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.289649 (+ 16924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.289650 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.289650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.289864 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.290074 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.290075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.290075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.290136 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.290408 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.290409 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.290455 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.290520 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.290624 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 07:14:30.291262 16850 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.130.252:38911: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a7e7537f3da642b095ac9def4634cb86"
member_type: NON_VOTER
last_known_addr {
host: "127.10.130.252"
port: 38911
}
attrs {
promote: true
}
}
dest_uuid: "433bb3767a8c44619896a663d6fcdf67"
cas_config_opid_index: 5
I20260924 07:14:30.291338 17592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.260648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:34245 (local address 127.0.0.1:40682)
0924 07:14:30.260951 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.260958 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.261029 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.261176 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.261177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.261404 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.261457 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.261458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.262319 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.262320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.263182 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.263184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.263634 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.263635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.263689 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.288426 (+ 24737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.288430 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.288809 (+ 379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.290367 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.290368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.290368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.290521 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.290714 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.290715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.290715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.290767 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.291053 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.291054 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.291095 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.291154 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.291246 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":72,"threads_started":1}
I20260924 07:14:30.291515 16850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } }
I20260924 07:14:30.291661 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.260825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40682 (local address 127.10.130.253:34245)
0924 07:14:30.260865 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.260866 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.260997 (+ 131us) server_negotiation.cc:407] Connection header received
0924 07:14:30.261081 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.261082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.261090 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.261129 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.261512 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.262230 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.263289 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.263576 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.263638 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.288891 (+ 25253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.288893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.288894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.288901 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.290298 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.290597 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.290598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.290599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.290655 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.290849 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.290849 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.290850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.290978 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.291031 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.291501 (+ 470us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.291585 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 07:14:30.292009 17408 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a7e7537f3da642b095ac9def4634cb86 (127.10.130.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } } }
I20260924 07:14:30.292743 17408 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.130.252:38911
I20260924 07:14:30.292843 17409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } } }
I20260924 07:14:30.292927 17409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:30.292976 17501 master.cc:537] Master@127.10.130.252:38911 shutting down...
I20260924 07:14:30.293154 17543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:30.294103 17543 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 1 FOLLOWER]: Refusing update from remote peer 433bb3767a8c44619896a663d6fcdf67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 07:14:30.294376 17409 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 07:14:30.294965 16837 consensus_peers.cc:487] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 -> Peer a7e7537f3da642b095ac9def4634cb86 (127.10.130.252:38911): Couldn't send request to peer a7e7537f3da642b095ac9def4634cb86. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 07:14:30.296182 17501 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:30.296264 17501 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:30.296314 17501 tablet_replica.cc:330] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: stopping tablet replica
I20260924 07:14:30.296311 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.291996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.10.130.253:34245)
0924 07:14:30.293093 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.293095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.293219 (+ 124us) server_negotiation.cc:407] Connection header received
0924 07:14:30.293238 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.293238 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.293258 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.293308 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:30.293455 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.294142 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.295449 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.295970 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.296059 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.296130 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.296225 (+ 95us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.296257 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1076}
I20260924 07:14:30.296314 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.291921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:34245 (local address 127.0.0.1:40694)
0924 07:14:30.293123 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.293132 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.293173 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.293356 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.293357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.293359 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:30.293395 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.293396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.294227 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.294228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.295363 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.295366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.296057 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.296058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.296125 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.296153 (+ 28us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.296218 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.296251 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1187}
I20260924 07:14:30.296567 17457 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 LEARNER]: Refusing update from remote peer 433bb3767a8c44619896a663d6fcdf67: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 07:14:30.296732 17409 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 07:14:30.297230 17457 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a7e7537f3da642b095ac9def4634cb86 (127.10.130.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } } }
W20260924 07:14:30.297499 17519 connection.cc:441] server connection from 127.0.0.1:41894 torn down before Call kudu.master.MasterService.ListTables from 127.0.0.1:41894 (request call id 4) could send its response
I20260924 07:14:30.298178 17595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 433bb3767a8c44619896a663d6fcdf67. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } } }
I20260924 07:14:30.298254 17595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: This master's current role is: LEARNER
I20260924 07:14:30.298211 17486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: NON_VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: true } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } } }
I20260924 07:14:30.298287 17486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: This master's current role is: LEARNER
I20260924 07:14:30.298619 17408 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67: attempting to promote NON_VOTER 5b151bdf2b7f4959b1d7bcb788a3c687 to VOTER
I20260924 07:14:30.298776 17408 consensus_queue.cc:237] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } }
I20260924 07:14:30.299134 17457 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 LEARNER]: Refusing update from remote peer 433bb3767a8c44619896a663d6fcdf67: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260924 07:14:30.299166 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.299068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38911 (local address 127.0.0.1:41934)
0924 07:14:30.299100 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.299131 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 07:14:30.299310 17596 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260924 07:14:30.299458 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.299323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38911 (local address 127.0.0.1:41936)
0924 07:14:30.299414 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.299434 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38911: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":59,"threads_started":1}
W20260924 07:14:30.299576 16837 consensus_peers.cc:487] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 -> Peer a7e7537f3da642b095ac9def4634cb86 (127.10.130.252:38911): Couldn't send request to peer a7e7537f3da642b095ac9def4634cb86. Status: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38911: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 07:14:30.299841 17409 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5b151bdf2b7f4959b1d7bcb788a3c687 (127.10.130.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } } }
I20260924 07:14:30.300547 17409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } } }
I20260924 07:14:30.300617 17409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:30.300577 17408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } } }
I20260924 07:14:30.300653 17408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:30.300714 17457 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5b151bdf2b7f4959b1d7bcb788a3c687 (127.10.130.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } } }
I20260924 07:14:30.301302 17486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } } }
I20260924 07:14:30.301378 17486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:14:30.309697 17501 master.cc:552] Master@127.10.130.252:38911 shutdown complete.
I20260924 07:14:30.309743 17501 master_runner.cc:305] Clearing existing system tablet
I20260924 07:14:30.309860 17501 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:30.309904 17501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:30.310536 17501 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:30.310971 17599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.311081 17501 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:30.311127 17501 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.311144 17501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/wal
uuid: "a7e7537f3da642b095ac9def4634cb86"
format_stamp: "Formatted at 2026-09-24 07:14:30 on dist-test-slave-7q7m"
server_key: "d1eb00ab43abba3df1167ea8ca2d91d8"
server_key_iv: "b30321b40f6b67dfa7d0e33b6fc29e93"
server_key_version: "encryptionkey@0"
I20260924 07:14:30.311702 17501 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:30.312867 17501 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 07:14:30.312919 17501 log.cc:1196] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260924 07:14:30.313068 17501 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: Deleting consensus metadata
I20260924 07:14:30.313160 17501 master_runner.cc:315] Copying system tablet from 127.10.130.254:40779
I20260924 07:14:30.313652 17501 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: tablet copy: Beginning tablet copy session from remote peer at address 127.10.130.254:40779
I20260924 07:14:30.318821 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.313930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35982 (local address 127.10.130.254:40779)
0924 07:14:30.313969 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.313971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.314041 (+ 70us) server_negotiation.cc:407] Connection header received
0924 07:14:30.314107 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.314108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.314117 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.314160 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.314410 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.314870 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.315510 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.315701 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.315750 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.316576 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.316577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.316578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.316584 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.317880 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.318171 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.318172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.318172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.318223 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.318382 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.318383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.318383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.318484 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.318538 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.318661 (+ 123us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.318728 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:14:30.318822 17605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.313895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:35982)
0924 07:14:30.313988 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.313997 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.314055 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.314162 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.314162 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.314315 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.314347 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.314348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.314932 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.314933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.315443 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.315445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.315755 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.315756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.315814 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.316245 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.316248 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.316501 (+ 253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.317956 (+ 1455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.317957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.317957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.318114 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.318274 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.318274 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.318275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.318335 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.318534 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.318535 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.318577 (+ 42us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.318636 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.318730 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":55,"threads_started":1}
I20260924 07:14:30.319239 16870 tablet_copy_service.cc:140] P 433bb3767a8c44619896a663d6fcdf67: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a7e7537f3da642b095ac9def4634cb86 ({username='oryx', principal='oryx/127.10.130.252@KRBTEST.COM'} at 127.0.0.1:35982)
I20260924 07:14:30.319298 16870 tablet_copy_service.cc:161] P 433bb3767a8c44619896a663d6fcdf67: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a7e7537f3da642b095ac9def4634cb86 at {username='oryx', principal='oryx/127.10.130.252@KRBTEST.COM'} at 127.0.0.1:35982: session id = a7e7537f3da642b095ac9def4634cb86-00000000000000000000000000000000
I20260924 07:14:30.319800 16870 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 07:14:30.320236 17501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 07:14:30.320996 17501 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: tablet copy: Starting download of 0 data blocks...
I20260924 07:14:30.321118 17501 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: tablet copy: Starting download of 1 WAL segments...
I20260924 07:14:30.321789 17501 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 07:14:30.322463 16870 tablet_copy_service.cc:342] P 433bb3767a8c44619896a663d6fcdf67: Request end of tablet copy session a7e7537f3da642b095ac9def4634cb86-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.130.252@KRBTEST.COM'} at 127.0.0.1:35982
I20260924 07:14:30.322520 16870 tablet_copy_service.cc:433] P 433bb3767a8c44619896a663d6fcdf67: ending tablet copy session a7e7537f3da642b095ac9def4634cb86-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a7e7537f3da642b095ac9def4634cb86
I20260924 07:14:30.324841 17501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:30.325824 17606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:30.325843 17607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:30.326025 17501 server_base.cc:1034] running on GCE node
W20260924 07:14:30.328605 17609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:30.328744 17501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:30.328869 17501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:30.329978 17501 hybrid_clock.cc:648] HybridClock initialized: now 1790234070329972 us; error 23 us; skew 500 ppm
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:30.331063 17501 init.cc:377] Logged in from keytab as oryx/127.10.130.252@KRBTEST.COM (short username oryx)
I20260924 07:14:30.331285 17501 webserver.cc:460] Webserver started at http://127.10.130.252:37203/ using document root <none> and password file <none>
I20260924 07:14:30.331398 17501 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:30.331429 17501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:30.331925 17501 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:30.332392 17616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.332489 17501 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.332530 17501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/wal
uuid: "a7e7537f3da642b095ac9def4634cb86"
format_stamp: "Formatted at 2026-09-24 07:14:30 on dist-test-slave-7q7m"
server_key: "d1eb00ab43abba3df1167ea8ca2d91d8"
server_key_iv: "b30321b40f6b67dfa7d0e33b6fc29e93"
server_key_version: "encryptionkey@0"
I20260924 07:14:30.332578 17501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:30.342199 17501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:30.343472 17501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:30.346069 17501 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.252:38911
I20260924 07:14:30.346184 17668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.252:38911 every 8 connection(s)
I20260924 07:14:30.346350 17501 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/master-2/data/info.pb
I20260924 07:14:30.347076 17669 sys_catalog.cc:263] Verifying existing consensus state
I20260924 07:14:30.347575 17669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: Bootstrap starting.
I20260924 07:14:30.350361 17669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 07:14:30.350556 17669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: Bootstrap complete.
I20260924 07:14:30.350760 17669 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } }
I20260924 07:14:30.350847 17669 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a7e7537f3da642b095ac9def4634cb86, State: Initialized, Role: LEARNER
I20260924 07:14:30.350884 17669 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } }
I20260924 07:14:30.351038 17669 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:30.351047 17673 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } } }
I20260924 07:14:30.351250 17673 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: This master's current role is: LEARNER
I20260924 07:14:30.353331 17684 catalog_manager.cc:1259] Loaded cluster ID: e06d2fbb994c47c180c5145efd658cc4
I20260924 07:14:30.353365 17684 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: loading cluster ID for follower catalog manager: success
I20260924 07:14:30.354364 17684 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: acquiring CA information for follower catalog manager: success
I20260924 07:14:30.354480 17671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.347966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41950 (local address 127.10.130.252:38911)
0924 07:14:30.348150 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.348151 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.348157 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:14:30.348172 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.348173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.348180 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.348229 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.348703 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.349458 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.350088 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.350291 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.350343 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.351606 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.351607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.351608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.351613 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.353184 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.353533 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.353534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.353534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.353594 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.353811 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.353812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.353812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.353925 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.354012 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.354302 (+ 290us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.354373 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":85,"threads_started":1}
I20260924 07:14:30.354701 17684 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 07:14:30.355002 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 16912
I20260924 07:14:30.359292 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:41189
--local_ip_for_outbound_sockets=127.10.130.193
--tserver_master_addrs=127.10.130.254:40779
--webserver_port=33719
--webserver_interface=127.10.130.193
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab
--principal=oryx/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.130.254:40779,127.10.130.253:34245
--tserver_master_addrs=127.10.130.254:40779,127.10.130.253:34245,127.10.130.252:38911 with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:30.439329 17685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:30.439510 17685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:30.439531 17685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:30.440666 17685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:30.440704 17685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:30.440728 17685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:30.442126 17685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab
--principal=oryx/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:41189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=33719
--tserver_master_addrs=127.10.130.254:40779,127.10.130.253:34245,127.10.130.252:38911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:30.442319 17685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:30.442530 17685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:30.447085 17691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:30.447136 17690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:30.447084 17693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:30.447320 17685 server_base.cc:1034] running on GCE node
I20260924 07:14:30.447523 17685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:30.447743 17685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:30.448910 17685 hybrid_clock.cc:648] HybridClock initialized: now 1790234070448891 us; error 40 us; skew 500 ppm
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:30.451246 17685 init.cc:377] Logged in from keytab as oryx/127.10.130.193@KRBTEST.COM (short username oryx)
I20260924 07:14:30.451699 17685 webserver.cc:460] Webserver started at http://127.10.130.193:33719/ using document root <none> and password file <none>
I20260924 07:14:30.451891 17685 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:30.451951 17685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:30.453299 17685 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:30.453873 17700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.454054 17685 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:30.454128 17685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "779bea9f583b47c09308fb3a22d42447"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "cb9114d8c7148266aeefa9594897d9f9"
server_key_iv: "c25a3281f15690f94c79f8db219c2461"
server_key_version: "encryptionkey@0"
I20260924 07:14:30.454432 17685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:30.467039 17685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:30.468598 17685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:30.468694 17685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:30.468845 17685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:30.469089 17685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:30.469120 17685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.469138 17685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:30.469161 17685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.474476 17685 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:41189
I20260924 07:14:30.474535 17813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:41189 every 8 connection(s)
I20260924 07:14:30.474810 17685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.252@KRBTEST.COM
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.253@KRBTEST.COM
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
I20260924 07:14:30.483458 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 17685
I20260924 07:14:30.483613 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 17046
I20260924 07:14:30.485244 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.475515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:59353 (local address 127.10.130.253:34245)
0924 07:14:30.475561 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.475563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.475829 (+ 266us) server_negotiation.cc:407] Connection header received
0924 07:14:30.476715 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.476717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.476729 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.476760 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.477375 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.478133 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.479882 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.480106 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.480159 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.482507 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.482508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.482508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.482515 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.483875 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.484427 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.484428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.484428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.484481 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.484687 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.484687 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.484688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.484799 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.484851 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.485079 (+ 228us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.485158 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:14:30.485260 17671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.475442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:50963 (local address 127.10.130.252:38911)
0924 07:14:30.475502 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.475504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.475711 (+ 207us) server_negotiation.cc:407] Connection header received
0924 07:14:30.476694 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.476695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.476710 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.476759 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.478231 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.478700 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.479440 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.479735 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.479792 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.482011 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.482013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.482013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.482020 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.483616 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.484404 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.484405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.484405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.484460 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.484693 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.484694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.484694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.484812 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.484858 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.485094 (+ 236us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.485173 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 07:14:30.485618 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.475394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:34245 (local address 127.10.130.193:59353)
0924 07:14:30.475768 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.475777 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.476613 (+ 836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.476838 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.476838 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.477133 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.477291 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.477297 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.478232 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.478232 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.479416 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.479419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.480168 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.480169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.480232 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.480543 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.480547 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.482443 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.484200 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.484201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.484202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.484365 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.484531 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.484532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.484532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.484629 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.484849 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.484854 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.484973 (+ 119us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.485054 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.485192 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":49,"thread_start_us":84,"threads_started":1}
I20260924 07:14:30.485623 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.475394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38911 (local address 127.10.130.193:50963)
0924 07:14:30.475603 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.475640 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.476598 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.476832 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.476836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.477117 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.477289 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.477294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.478771 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.478772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.479365 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.479367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.479809 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.479810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.479946 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.480438 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.480447 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.481935 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.483698 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.484032 (+ 334us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.484034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.484320 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.484524 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.484525 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.484525 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.484630 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.484887 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.484887 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.484979 (+ 92us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.485370 (+ 391us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.485456 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":86,"threads_started":1}
I20260924 07:14:30.486132 17814 heartbeater.cc:347] Connected to a master server at 127.10.130.252:38911
I20260924 07:14:30.486097 17816 heartbeater.cc:347] Connected to a master server at 127.10.130.253:34245
I20260924 07:14:30.486209 17814 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:30.486209 17816 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:30.486392 17816 heartbeater.cc:511] Master 127.10.130.253:34245 requested a full tablet report, sending...
I20260924 07:14:30.486392 17814 heartbeater.cc:511] Master 127.10.130.252:38911 requested a full tablet report, sending...
I20260924 07:14:30.486845 17633 ts_manager.cc:194] Registered new tserver with Master: 779bea9f583b47c09308fb3a22d42447 (127.10.130.193:41189)
I20260924 07:14:30.487076 17447 ts_manager.cc:194] Registered new tserver with Master: 779bea9f583b47c09308fb3a22d42447 (127.10.130.193:41189)
I20260924 07:14:30.489197 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:38631
--local_ip_for_outbound_sockets=127.10.130.194
--tserver_master_addrs=127.10.130.254:40779
--webserver_port=39847
--webserver_interface=127.10.130.194
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab
--principal=oryx/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.130.254:40779,127.10.130.253:34245
--tserver_master_addrs=127.10.130.254:40779,127.10.130.253:34245,127.10.130.252:38911 with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:14:30.490101 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.475742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:39209 (local address 127.10.130.254:40779)
0924 07:14:30.475787 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.475789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.475837 (+ 48us) server_negotiation.cc:407] Connection header received
0924 07:14:30.476722 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.476724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.476735 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.476762 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.477395 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.478137 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.479486 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.479770 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.479836 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.483036 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.483038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.483038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.483045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.489229 (+ 6184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.489548 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.489548 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.489549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.489597 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.489747 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.489747 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.489747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.489837 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.489880 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.489994 (+ 114us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.490047 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 07:14:30.490147 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.475675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.193:39209)
0924 07:14:30.475785 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.475794 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.476616 (+ 822us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.476838 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.476838 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.477126 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.477291 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.477297 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.478232 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.478233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.479416 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.479419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.479833 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.479833 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.479946 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.480760 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.480765 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.482935 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.489309 (+ 6374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.489310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.489311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.489483 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.489646 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.489646 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.489647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.489700 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.489882 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.489883 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.489919 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.489979 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.490063 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":55,"thread_start_us":56,"threads_started":1}
I20260924 07:14:30.490700 17815 heartbeater.cc:347] Connected to a master server at 127.10.130.254:40779
I20260924 07:14:30.490746 17815 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:30.490790 17815 heartbeater.cc:511] Master 127.10.130.254:40779 requested a full tablet report, sending...
I20260924 07:14:30.491019 16850 ts_manager.cc:194] Re-registered known tserver with Master: 779bea9f583b47c09308fb3a22d42447 (127.10.130.193:41189)
I20260924 07:14:30.491465 16850 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:39209
W20260924 07:14:30.566987 17821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:30.567178 17821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:30.567198 17821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:30.568370 17821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:30.568408 17821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:30.568437 17821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:30.569861 17821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab
--principal=oryx/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:38631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=39847
--tserver_master_addrs=127.10.130.254:40779,127.10.130.253:34245,127.10.130.252:38911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:30.570075 17821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:30.570278 17821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:30.574950 17829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:30.574946 17826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:30.575040 17827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:30.575203 17821 server_base.cc:1034] running on GCE node
I20260924 07:14:30.575349 17821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:30.575567 17821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:30.576717 17821 hybrid_clock.cc:648] HybridClock initialized: now 1790234070576706 us; error 26 us; skew 500 ppm
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:30.578902 17821 init.cc:377] Logged in from keytab as oryx/127.10.130.194@KRBTEST.COM (short username oryx)
I20260924 07:14:30.579283 17821 webserver.cc:460] Webserver started at http://127.10.130.194:39847/ using document root <none> and password file <none>
I20260924 07:14:30.579473 17821 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:30.579524 17821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:30.580762 17821 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:30.581260 17836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.581387 17821 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.581470 17821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "bdc83d87f36a4efeb4076c1e3c68917e"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "abab960cfdaee016a769b1e9b4b40782"
server_key_iv: "af7ef700cc04d2ad10e4a7e25de2b955"
server_key_version: "encryptionkey@0"
I20260924 07:14:30.581768 17821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:30.596107 17821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:30.598002 17821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:30.598121 17821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:30.598320 17821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:30.598583 17821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:30.598614 17821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.598641 17821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:30.598658 17821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.603859 17821 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:38631
I20260924 07:14:30.603916 17949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:38631 every 8 connection(s)
I20260924 07:14:30.604219 17821 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:14:30.604549 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 17821
I20260924 07:14:30.604707 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 17180
I20260924 07:14:30.610379 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:38491
--local_ip_for_outbound_sockets=127.10.130.195
--tserver_master_addrs=127.10.130.254:40779
--webserver_port=42319
--webserver_interface=127.10.130.195
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab
--principal=oryx/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.130.254:40779,127.10.130.253:34245
--tserver_master_addrs=127.10.130.254:40779,127.10.130.253:34245,127.10.130.252:38911 with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.252@KRBTEST.COM
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.253@KRBTEST.COM
I20260924 07:14:30.616312 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.605376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:58383 (local address 127.10.130.254:40779)
0924 07:14:30.605415 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.605417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.606906 (+ 1489us) server_negotiation.cc:407] Connection header received
0924 07:14:30.607000 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.607001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.607013 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.607039 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.607431 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.608190 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.609596 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.609883 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.609923 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.613027 (+ 3104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.613028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.613029 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.613037 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.614795 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.615288 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.615290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.615290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.615347 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.615611 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.615612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.615612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.615748 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.615791 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.616152 (+ 361us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.616226 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:14:30.616343 17671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.606296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:57575 (local address 127.10.130.252:38911)
0924 07:14:30.606367 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.606369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.606378 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:30.606405 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.606406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.606417 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.606451 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.607260 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.608056 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.609595 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.609895 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.609930 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.613375 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.613376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.613377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.613383 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.614886 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.615296 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.615297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.615298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.615365 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.615620 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.615621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.615621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.615749 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.615790 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.616162 (+ 372us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.616238 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 07:14:30.616684 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.605154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:43999 (local address 127.10.130.253:34245)
0924 07:14:30.605206 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.605208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.605536 (+ 328us) server_negotiation.cc:407] Connection header received
0924 07:14:30.606559 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.606561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.606572 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.606599 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.607260 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.608056 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.609594 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.609893 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.609930 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.613560 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.613561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.613561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.613568 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.615564 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.615880 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.615881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.615881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.615931 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.616142 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.616143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.616143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.616267 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.616307 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.616539 (+ 232us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.616603 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 07:14:30.616770 17956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.605313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.194:58383)
0924 07:14:30.606840 (+ 1527us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.606850 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.606933 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.607096 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.607097 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.607258 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.607329 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.607331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.608279 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.608280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.609496 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.609499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.610004 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.610004 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.610132 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.610769 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.610774 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.612932 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.614877 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.614884 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.614886 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.615202 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.615415 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.615416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.615416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.615539 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.615834 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.615840 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.616005 (+ 165us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.616395 (+ 390us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.616548 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1510,"mutex_wait_us":74,"thread_start_us":62,"threads_started":1}
I20260924 07:14:30.617074 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.604963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:34245 (local address 127.10.130.194:43999)
0924 07:14:30.605451 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.605463 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.606482 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.606685 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.606686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.606893 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.607159 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.607164 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.608181 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.608181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.609497 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.609500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.610010 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.610010 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.610134 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.610871 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.610875 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.613491 (+ 2616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.615634 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.615634 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.615635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.615807 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.616021 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.616021 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.616021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.616074 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.616351 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.616352 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.616430 (+ 78us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.616784 (+ 354us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.616869 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":117,"thread_start_us":259,"threads_started":1}
I20260924 07:14:30.617154 17951 heartbeater.cc:347] Connected to a master server at 127.10.130.254:40779
I20260924 07:14:30.617221 17951 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:30.617409 17952 heartbeater.cc:347] Connected to a master server at 127.10.130.253:34245
I20260924 07:14:30.617386 17951 heartbeater.cc:511] Master 127.10.130.254:40779 requested a full tablet report, sending...
I20260924 07:14:30.617461 17952 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:30.617516 17952 heartbeater.cc:511] Master 127.10.130.253:34245 requested a full tablet report, sending...
I20260924 07:14:30.617529 17955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.604962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38911 (local address 127.10.130.194:57575)
0924 07:14:30.605281 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.605463 (+ 182us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.606326 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.606530 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.606535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.606893 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.607159 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.607164 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.608183 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.608183 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.609497 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.609500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.610078 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.610079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.610134 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.610724 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.610734 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.613297 (+ 2563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.614954 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.614955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.614955 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.615207 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.615438 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.615439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.615439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.615546 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.615837 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.615843 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.616016 (+ 173us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.617010 (+ 994us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.617327 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":90,"thread_start_us":167,"threads_started":1}
I20260924 07:14:30.617821 17447 ts_manager.cc:194] Registered new tserver with Master: bdc83d87f36a4efeb4076c1e3c68917e (127.10.130.194:38631)
I20260924 07:14:30.617959 16850 ts_manager.cc:194] Re-registered known tserver with Master: bdc83d87f36a4efeb4076c1e3c68917e (127.10.130.194:38631)
I20260924 07:14:30.618027 17950 heartbeater.cc:347] Connected to a master server at 127.10.130.252:38911
I20260924 07:14:30.618075 17950 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:30.618142 17950 heartbeater.cc:511] Master 127.10.130.252:38911 requested a full tablet report, sending...
I20260924 07:14:30.618398 17633 ts_manager.cc:194] Registered new tserver with Master: bdc83d87f36a4efeb4076c1e3c68917e (127.10.130.194:38631)
I20260924 07:14:30.618434 16850 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:58383
W20260924 07:14:30.690171 17957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:30.690347 17957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:30.690367 17957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:30.691431 17957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:30.691463 17957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:30.691488 17957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:30.692906 17957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab
--principal=oryx/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:38491
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=42319
--tserver_master_addrs=127.10.130.254:40779,127.10.130.253:34245,127.10.130.252:38911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:30.693078 17957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:30.693244 17957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:30.697674 17962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:30.697721 17963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:30.697695 17965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:30.697780 17957 server_base.cc:1034] running on GCE node
I20260924 07:14:30.698060 17957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:30.698268 17957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:30.699409 17957 hybrid_clock.cc:648] HybridClock initialized: now 1790234070699395 us; error 26 us; skew 500 ppm
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:30.701596 17957 init.cc:377] Logged in from keytab as oryx/127.10.130.195@KRBTEST.COM (short username oryx)
I20260924 07:14:30.701956 17957 webserver.cc:460] Webserver started at http://127.10.130.195:42319/ using document root <none> and password file <none>
I20260924 07:14:30.702149 17957 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:30.702208 17957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:30.703406 17957 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:30.703915 17972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.704061 17957 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.704134 17957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "e25329d24dfa425f8984014b7813fea8"
format_stamp: "Formatted at 2026-09-24 07:14:29 on dist-test-slave-7q7m"
server_key: "0c89d368f04caef54a6ca9b06509973d"
server_key_iv: "f1b6e507a4926a78f13c324a3235fc65"
server_key_version: "encryptionkey@0"
I20260924 07:14:30.704434 17957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:30.716078 17671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.713352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41962 (local address 127.10.130.252:38911)
0924 07:14:30.713416 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.713419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.713428 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:30.713454 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.713455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.713477 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.713530 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:30.713657 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.714381 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.715264 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.715803 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.715871 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.715924 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.715963 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.716013 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 07:14:30.716090 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.713268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38911 (local address 127.0.0.1:41962)
0924 07:14:30.713316 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.713328 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.713387 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.713549 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.713550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.713551 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:30.713599 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.713601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.714437 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.714438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.715194 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.715197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.715874 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.715875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.715918 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.715922 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.715958 (+ 36us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.716037 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260924 07:14:30.716396 17643 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260924 07:14:30.716529 17673 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 433bb3767a8c44619896a663d6fcdf67. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } } }
I20260924 07:14:30.716642 17673 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: This master's current role is: LEARNER
I20260924 07:14:30.718930 17957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:30.720574 17957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:30.720690 17957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:30.720862 17957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:30.721110 17957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:30.721143 17957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.721167 17957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:30.721184 17957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:30.726287 17957 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:38491
I20260924 07:14:30.726349 18085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:38491 every 8 connection(s)
I20260924 07:14:30.726619 17957 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.253@KRBTEST.COM
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
Sep 24 07:14:30 dist-test-slave-7q7m krb5kdc[16805](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234070, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.252@KRBTEST.COM
I20260924 07:14:30.734781 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 17957
I20260924 07:14:30.737406 18091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.727188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38911 (local address 127.10.130.195:54799)
0924 07:14:30.727350 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.727386 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.727884 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.728148 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.728151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.728451 (+ 300us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.728646 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.728651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.730101 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.730102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.730718 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.730720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.731418 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.731419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.731479 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.731819 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.731823 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.734289 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.735962 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.735963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.735963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.736191 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.736412 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.736413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.736413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.736536 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.736804 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.736809 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.736951 (+ 142us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.737045 (+ 94us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.737192 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":2,"spinlock_wait_cycles":384,"thread_start_us":87,"threads_started":1}
I20260924 07:14:30.737824 17671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.727208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:54799 (local address 127.10.130.252:38911)
0924 07:14:30.727254 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.727256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.727487 (+ 231us) server_negotiation.cc:407] Connection header received
0924 07:14:30.727970 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.727990 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.728003 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.728053 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.729312 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.730014 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.730799 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.731336 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.731401 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.734394 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.734395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.734396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.734403 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.735871 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.736269 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.736270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.736270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.736332 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.736601 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.736602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.736602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.736724 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.736780 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.737674 (+ 894us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.737741 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 07:14:30.737835 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.727591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:44233 (local address 127.10.130.254:40779)
0924 07:14:30.727645 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.727647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.727657 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:14:30.728014 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.728015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.728026 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.728055 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.728753 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.729213 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.730321 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.730505 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.730557 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.734402 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.734403 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.734404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.734410 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.736574 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.736911 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.736912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.736913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.736966 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.737138 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.737139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.737139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.737253 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.737308 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.737674 (+ 366us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.737741 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 07:14:30.737610 18092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.727471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.195:44233)
0924 07:14:30.727592 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.727601 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.727892 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.728444 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.728444 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.728573 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.728646 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.728652 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.729307 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.729307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.730235 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.730237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.730570 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.730571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.730698 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.731112 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.731119 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.734289 (+ 3170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.736645 (+ 2356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.736645 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.736646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.736843 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.737026 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.737027 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.737027 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.737080 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.737332 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.737333 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.737369 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.737437 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.737527 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":78,"thread_start_us":74,"threads_started":1}
I20260924 07:14:30.738732 18086 heartbeater.cc:347] Connected to a master server at 127.10.130.252:38911
I20260924 07:14:30.738723 18087 heartbeater.cc:347] Connected to a master server at 127.10.130.254:40779
I20260924 07:14:30.738842 18086 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:30.738960 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.727170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:56209 (local address 127.10.130.253:34245)
0924 07:14:30.727280 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.727282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.727466 (+ 184us) server_negotiation.cc:407] Connection header received
0924 07:14:30.728039 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.728041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.728052 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.728080 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.728747 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.729212 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.730607 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.730809 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.730869 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.732958 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.732960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.732961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.732968 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.734270 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.735820 (+ 1550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.735821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.735822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.735881 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.738039 (+ 2158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.738040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.738041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.738160 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.738212 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.738790 (+ 578us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.738874 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260924 07:14:30.739027 18086 heartbeater.cc:511] Master 127.10.130.252:38911 requested a full tablet report, sending...
I20260924 07:14:30.738724 18090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.727188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:34245 (local address 127.10.130.195:56209)
0924 07:14:30.727334 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.727370 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.727877 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.728147 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.728153 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.728460 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:30.728647 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.728652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.729315 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.729316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.730246 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.730248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.730878 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.730879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.730943 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.731208 (+ 265us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:30.731212 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:30.732880 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:30.734346 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.734352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.734353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.734612 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.737891 (+ 3279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:30.737892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:30.737893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:30.737957 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:30.738273 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:30.738274 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:30.738312 (+ 38us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.738363 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.738448 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":31,"thread_start_us":80,"threads_started":1}
I20260924 07:14:30.738842 18087 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:30.739223 18087 heartbeater.cc:511] Master 127.10.130.254:40779 requested a full tablet report, sending...
I20260924 07:14:30.739327 18088 heartbeater.cc:347] Connected to a master server at 127.10.130.253:34245
I20260924 07:14:30.739368 18088 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:30.739434 18088 heartbeater.cc:511] Master 127.10.130.253:34245 requested a full tablet report, sending...
I20260924 07:14:30.739494 16850 ts_manager.cc:194] Re-registered known tserver with Master: e25329d24dfa425f8984014b7813fea8 (127.10.130.195:38491)
I20260924 07:14:30.739598 17633 ts_manager.cc:194] Registered new tserver with Master: e25329d24dfa425f8984014b7813fea8 (127.10.130.195:38491)
I20260924 07:14:30.739678 17447 ts_manager.cc:194] Registered new tserver with Master: e25329d24dfa425f8984014b7813fea8 (127.10.130.195:38491)
I20260924 07:14:30.739940 16850 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:44233
I20260924 07:14:30.744045 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.736073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.10.130.252:38911)
0924 07:14:30.736182 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.736183 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.736990 (+ 807us) server_negotiation.cc:407] Connection header received
0924 07:14:30.737013 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.737014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.737027 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.737085 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.737801 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.738456 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.739431 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.739834 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.739890 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.741489 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.741490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.741491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.741496 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.742916 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.743301 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.743302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.743303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.743364 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.743563 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.743564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.743564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.743683 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.743773 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.743858 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.743929 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":66,"threads_started":1}
I20260924 07:14:30.744819 18097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.735704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35988 (local address 127.10.130.254:40779)
0924 07:14:30.736069 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.736071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.736367 (+ 296us) server_negotiation.cc:407] Connection header received
0924 07:14:30.736467 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.736468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.736484 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.736532 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.737394 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.738105 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.740652 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.740871 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.740937 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.742119 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.742120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.742121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.742127 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.743540 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.743914 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.743915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.743915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.743999 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.744231 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.744232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.744232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.744355 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.744428 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.744643 (+ 215us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.744718 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":77,"threads_started":1}
I20260924 07:14:30.746876 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.736561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40708 (local address 127.10.130.253:34245)
0924 07:14:30.736824 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.736826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.740021 (+ 3195us) server_negotiation.cc:407] Connection header received
0924 07:14:30.740105 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.740105 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.740116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.740165 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.741140 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.741824 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.743137 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.743374 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.743426 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.744704 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.744705 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.744705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.744712 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.745930 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.746207 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.746207 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.746208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.746260 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.746431 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.746432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.746433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.746545 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.746638 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.746716 (+ 78us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.746784 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":53,"threads_started":1}
I20260924 07:14:30.746390 16850 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:14:30.748239 16850 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:30.748831 17596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: NON_VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: true } } }
I20260924 07:14:30.748934 17596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:30.750015 17596 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67: attempting to promote NON_VOTER a7e7537f3da642b095ac9def4634cb86 to VOTER
I20260924 07:14:30.750129 17596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } }
I20260924 07:14:30.750459 17457 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 FOLLOWER]: Refusing update from remote peer 433bb3767a8c44619896a663d6fcdf67: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260924 07:14:30.750869 17642 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 1 LEARNER]: Refusing update from remote peer 433bb3767a8c44619896a663d6fcdf67: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260924 07:14:30.750874 18104 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 07:14:30.751098 18104 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260924 07:14:30.751546 17409 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a7e7537f3da642b095ac9def4634cb86 (127.10.130.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } } }
I20260924 07:14:30.751706 17457 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a7e7537f3da642b095ac9def4634cb86 (127.10.130.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } } }
I20260924 07:14:30.752076 17642 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, a7e7537f3da642b095ac9def4634cb86 (127.10.130.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } } }
I20260924 07:14:30.752418 17409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } } }
I20260924 07:14:30.752491 17409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:30.752537 17409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } } }
I20260924 07:14:30.752579 17409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:30.752615 17409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } } }
I20260924 07:14:30.752656 17409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:30.752710 17486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } } }
I20260924 07:14:30.752796 17486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:14:30.753150 17673 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "433bb3767a8c44619896a663d6fcdf67" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } } }
I20260924 07:14:30.753237 17673 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:14:30.757261 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.753636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:38631 (local address 127.0.0.1:46916)
0924 07:14:30.753683 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.753693 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.753734 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.754137 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.754138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.754140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:30.754175 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.754176 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.755548 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.755549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.756392 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.756394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.757054 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.757055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.757110 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.757113 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.757165 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.757201 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 07:14:30.757478 18111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.753723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46916 (local address 127.10.130.194:38631)
0924 07:14:30.753966 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.753970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.753980 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:14:30.754018 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.754020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.754036 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.754241 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:30.754296 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.755483 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.756464 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.756984 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.757095 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.757189 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.757215 (+ 26us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.757255 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":190,"threads_started":1}
I20260924 07:14:30.757665 17597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.753681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38491 (local address 127.0.0.1:43370)
0924 07:14:30.753786 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.753797 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.753844 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.754202 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.754203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.754205 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:30.754249 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.754251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.755460 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.755461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.756577 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.756580 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.757433 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.757434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.757492 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.757495 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.757526 (+ 31us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.757575 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260924 07:14:30.758013 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.753919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43370 (local address 127.10.130.195:38491)
0924 07:14:30.754049 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.754052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.754062 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:14:30.754087 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.754089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.754102 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.754194 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:30.754334 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.755358 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.756684 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.757345 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.757831 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.757885 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.757910 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.757944 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260924 07:14:30.758213 17884 tablet_service.cc:1467] Processing CreateTablet for tablet d2403bedfe4c40e09c44ac6ce85d4742 (DEFAULT_TABLE table=test-table [id=2d354cf3e1e3478493485251182710fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:30.758371 18114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.753741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:41189 (local address 127.0.0.1:38448)
0924 07:14:30.754195 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.754204 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.754262 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.754436 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.754437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.754439 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:30.754472 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.754473 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.756104 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.756104 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.757194 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.757196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.758150 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.758151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.758213 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.758217 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.758270 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.758306 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":399,"threads_started":1}
I20260924 07:14:30.758558 17884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2403bedfe4c40e09c44ac6ce85d4742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:30.758612 18113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.753980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38448 (local address 127.10.130.193:41189)
0924 07:14:30.754120 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.754124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.754264 (+ 140us) server_negotiation.cc:407] Connection header received
0924 07:14:30.754334 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.754336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.754352 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.754427 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:30.754564 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.755969 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.757335 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.758015 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.758114 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.758181 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.758496 (+ 315us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.758537 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260924 07:14:30.758565 18020 tablet_service.cc:1467] Processing CreateTablet for tablet d2403bedfe4c40e09c44ac6ce85d4742 (DEFAULT_TABLE table=test-table [id=2d354cf3e1e3478493485251182710fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:30.758854 18020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2403bedfe4c40e09c44ac6ce85d4742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:30.761165 18115 tablet_bootstrap.cc:492] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e: Bootstrap starting.
I20260924 07:14:30.761847 18115 tablet_bootstrap.cc:654] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:30.761861 18116 tablet_bootstrap.cc:492] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8: Bootstrap starting.
I20260924 07:14:30.761760 17748 tablet_service.cc:1467] Processing CreateTablet for tablet d2403bedfe4c40e09c44ac6ce85d4742 (DEFAULT_TABLE table=test-table [id=2d354cf3e1e3478493485251182710fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:30.762104 17748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2403bedfe4c40e09c44ac6ce85d4742. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:30.762136 18115 log.cc:824] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:30.762422 18116 tablet_bootstrap.cc:654] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:30.762741 18116 log.cc:824] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:30.762778 18115 tablet_bootstrap.cc:492] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e: No bootstrap required, opened a new log
I20260924 07:14:30.762823 18115 ts_tablet_manager.cc:1397] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:30.763497 18116 tablet_bootstrap.cc:492] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8: No bootstrap required, opened a new log
I20260924 07:14:30.763551 18116 ts_tablet_manager.cc:1397] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:30.764879 18115 raft_consensus.cc:348] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } } peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } }
I20260924 07:14:30.765003 18115 raft_consensus.cc:374] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:30.765038 18115 raft_consensus.cc:729] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdc83d87f36a4efeb4076c1e3c68917e, State: Initialized, Role: FOLLOWER
I20260924 07:14:30.765173 18115 consensus_queue.cc:260] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } } peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } }
I20260924 07:14:30.765436 18115 ts_tablet_manager.cc:1428] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:14:30.765384 18116 raft_consensus.cc:348] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } } peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } }
I20260924 07:14:30.765529 17951 heartbeater.cc:503] Master 127.10.130.254:40779 was elected leader, sending a full tablet report...
I20260924 07:14:30.765537 18116 raft_consensus.cc:374] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:30.765571 18116 raft_consensus.cc:729] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e25329d24dfa425f8984014b7813fea8, State: Initialized, Role: FOLLOWER
I20260924 07:14:30.765669 18116 consensus_queue.cc:260] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } } peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } }
I20260924 07:14:30.765890 18116 ts_tablet_manager.cc:1428] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 07:14:30.766216 18087 heartbeater.cc:503] Master 127.10.130.254:40779 was elected leader, sending a full tablet report...
I20260924 07:14:30.766824 18119 tablet_bootstrap.cc:492] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447: Bootstrap starting.
I20260924 07:14:30.767311 18119 tablet_bootstrap.cc:654] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:30.767578 18119 log.cc:824] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:30.768302 18119 tablet_bootstrap.cc:492] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447: No bootstrap required, opened a new log
I20260924 07:14:30.768356 18119 ts_tablet_manager.cc:1397] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:30.769604 18119 raft_consensus.cc:348] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } } peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } }
I20260924 07:14:30.769717 18119 raft_consensus.cc:374] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:30.769737 18119 raft_consensus.cc:729] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779bea9f583b47c09308fb3a22d42447, State: Initialized, Role: FOLLOWER
I20260924 07:14:30.769819 18119 consensus_queue.cc:260] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } } peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } }
I20260924 07:14:30.770007 18119 ts_tablet_manager.cc:1428] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:14:30.770076 17815 heartbeater.cc:503] Master 127.10.130.254:40779 was elected leader, sending a full tablet report...
I20260924 07:14:30.788245 18123 raft_consensus.cc:482] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:30.788326 18123 raft_consensus.cc:504] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } } peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } }
I20260924 07:14:30.788602 18123 leader_election.cc:288] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdc83d87f36a4efeb4076c1e3c68917e (127.10.130.194:38631), e25329d24dfa425f8984014b7813fea8 (127.10.130.195:38491)
I20260924 07:14:30.790946 18112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.788831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:59341 (local address 127.10.130.195:38491)
0924 07:14:30.788869 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.788871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.788877 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:14:30.788897 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.788898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.788905 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.788948 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:30.789067 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.789500 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.790344 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.790673 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.790729 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.790736 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.790862 (+ 126us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.790895 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:14:30.790947 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.788760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38491 (local address 127.10.130.193:59341)
0924 07:14:30.788821 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.788830 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.788859 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.788955 (+ 96us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.788956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.788957 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:30.789014 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.789015 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.789564 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.789565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.790256 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.790258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.790736 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.790737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.790797 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.790799 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.790848 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.790880 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":2}
I20260924 07:14:30.790997 18111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.788761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:38227 (local address 127.10.130.194:38631)
0924 07:14:30.788799 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.788801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.788833 (+ 32us) server_negotiation.cc:407] Connection header received
0924 07:14:30.788855 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.788856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.788865 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.788918 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:30.789056 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.789507 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.790361 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.790699 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.790763 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.790770 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.790908 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.790941 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 07:14:30.791042 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.788711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:38631 (local address 127.10.130.193:38227)
0924 07:14:30.788748 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:30.788758 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:30.788795 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:30.788931 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:30.788932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:30.788934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:30.788994 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.788995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.789578 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.789579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.790273 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:30.790275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.790767 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.790768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:30.790843 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.790846 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:30.790896 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 07:14:30.790934 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 07:14:30.791240 18040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2403bedfe4c40e09c44ac6ce85d4742" candidate_uuid: "779bea9f583b47c09308fb3a22d42447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e25329d24dfa425f8984014b7813fea8" is_pre_election: true
I20260924 07:14:30.791332 17904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2403bedfe4c40e09c44ac6ce85d4742" candidate_uuid: "779bea9f583b47c09308fb3a22d42447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" is_pre_election: true
I20260924 07:14:30.791364 18040 raft_consensus.cc:2454] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 779bea9f583b47c09308fb3a22d42447 in term 0.
I20260924 07:14:30.791435 17904 raft_consensus.cc:2454] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 779bea9f583b47c09308fb3a22d42447 in term 0.
I20260924 07:14:30.791567 17701 leader_election.cc:302] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 779bea9f583b47c09308fb3a22d42447, e25329d24dfa425f8984014b7813fea8; no voters:
I20260924 07:14:30.791688 18123 raft_consensus.cc:2781] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:30.791738 18123 raft_consensus.cc:482] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:30.791754 18123 raft_consensus.cc:3037] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:30.792342 18123 raft_consensus.cc:504] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } } peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } }
I20260924 07:14:30.792454 18123 leader_election.cc:288] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [CANDIDATE]: Term 1 election: Requested vote from peers bdc83d87f36a4efeb4076c1e3c68917e (127.10.130.194:38631), e25329d24dfa425f8984014b7813fea8 (127.10.130.195:38491)
I20260924 07:14:30.792644 17904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2403bedfe4c40e09c44ac6ce85d4742" candidate_uuid: "779bea9f583b47c09308fb3a22d42447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdc83d87f36a4efeb4076c1e3c68917e"
I20260924 07:14:30.792734 17904 raft_consensus.cc:3037] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:30.792878 18040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2403bedfe4c40e09c44ac6ce85d4742" candidate_uuid: "779bea9f583b47c09308fb3a22d42447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e25329d24dfa425f8984014b7813fea8"
I20260924 07:14:30.792968 18040 raft_consensus.cc:3037] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:30.793339 17904 raft_consensus.cc:2454] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 779bea9f583b47c09308fb3a22d42447 in term 1.
I20260924 07:14:30.793529 17703 leader_election.cc:302] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 779bea9f583b47c09308fb3a22d42447, bdc83d87f36a4efeb4076c1e3c68917e; no voters:
I20260924 07:14:30.793603 18123 raft_consensus.cc:2781] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:30.793661 18040 raft_consensus.cc:2454] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 779bea9f583b47c09308fb3a22d42447 in term 1.
I20260924 07:14:30.793720 18123 raft_consensus.cc:686] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [term 1 LEADER]: Becoming Leader. State: Replica: 779bea9f583b47c09308fb3a22d42447, State: Running, Role: LEADER
I20260924 07:14:30.793792 18123 consensus_queue.cc:237] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } } peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } }
I20260924 07:14:30.794437 16850 catalog_manager.cc:5495] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 reported cstate change: term changed from 0 to 1, leader changed from <none> to 779bea9f583b47c09308fb3a22d42447 (127.10.130.193). New cstate: current_term: 1 leader_uuid: "779bea9f583b47c09308fb3a22d42447" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } health_report { overall_health: HEALTHY } } }
I20260924 07:14:30.800436 18113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.798378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38454 (local address 127.10.130.193:41189)
0924 07:14:30.798411 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.798413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.798421 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:30.798475 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.798476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.798491 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.798546 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:30.798659 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.799082 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.799785 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.799968 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.800032 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.800185 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:30.800263 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:30.800327 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.800362 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.800378 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:14:30.801728 17904 raft_consensus.cc:1261] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e [term 1 FOLLOWER]: Refusing update from remote peer 779bea9f583b47c09308fb3a22d42447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:30.801733 18040 raft_consensus.cc:1261] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8 [term 1 FOLLOWER]: Refusing update from remote peer 779bea9f583b47c09308fb3a22d42447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:30.801980 18124 consensus_queue.cc:1035] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:30.802090 18123 consensus_queue.cc:1035] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:30.804675 18129 mvcc.cc:205] Tried to move back new op lower bound from 7332798754002014208 to 7332798753971798016. Current Snapshot: MvccSnapshot[applied={T|T < 7332798754002014208}]
I20260924 07:14:30.805423 18128 mvcc.cc:205] Tried to move back new op lower bound from 7332798754002014208 to 7332798753971798016. Current Snapshot: MvccSnapshot[applied={T|T < 7332798754002014208}]
I20260924 07:14:30.806972 16850 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:30.807044 16850 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:30.808465 16850 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 433bb3767a8c44619896a663d6fcdf67: Sending DeleteTablet for 3 replicas of tablet d2403bedfe4c40e09c44ac6ce85d4742
I20260924 07:14:30.808729 18020 tablet_service.cc:1514] Processing DeleteTablet for tablet d2403bedfe4c40e09c44ac6ce85d4742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:30 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:43370
I20260924 07:14:30.808782 17884 tablet_service.cc:1514] Processing DeleteTablet for tablet d2403bedfe4c40e09c44ac6ce85d4742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:30 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:46916
I20260924 07:14:30.808930 18136 tablet_replica.cc:330] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8: stopping tablet replica
I20260924 07:14:30.808923 17748 tablet_service.cc:1514] Processing DeleteTablet for tablet d2403bedfe4c40e09c44ac6ce85d4742 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:30 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:38448
I20260924 07:14:30.809029 18136 raft_consensus.cc:2229] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:30.809090 18138 tablet_replica.cc:330] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447: stopping tablet replica
I20260924 07:14:30.809126 18136 raft_consensus.cc:2258] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:30.809168 18138 raft_consensus.cc:2229] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:30.809269 18138 raft_consensus.cc:2258] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:30.809402 18137 tablet_replica.cc:330] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e: stopping tablet replica
I20260924 07:14:30.809484 18137 raft_consensus.cc:2229] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:30.809549 18136 ts_tablet_manager.cc:1905] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:30.809568 18137 raft_consensus.cc:2258] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:30.809688 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 16818
I20260924 07:14:30.810087 18137 ts_tablet_manager.cc:1905] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:30.811259 18137 ts_tablet_manager.cc:1918] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:30.811314 18137 log.cc:1196] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/d2403bedfe4c40e09c44ac6ce85d4742
I20260924 07:14:30.811515 18137 ts_tablet_manager.cc:1939] T d2403bedfe4c40e09c44ac6ce85d4742 P bdc83d87f36a4efeb4076c1e3c68917e: Deleting consensus metadata
I20260924 07:14:30.813097 18136 ts_tablet_manager.cc:1918] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:30.813151 18136 log.cc:1196] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/d2403bedfe4c40e09c44ac6ce85d4742
I20260924 07:14:30.813350 18136 ts_tablet_manager.cc:1939] T d2403bedfe4c40e09c44ac6ce85d4742 P e25329d24dfa425f8984014b7813fea8: Deleting consensus metadata
W20260924 07:14:30.816371 17976 connection.cc:535] server connection from 127.0.0.1:43370 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 07:14:30.816474 17839 connection.cc:535] server connection from 127.0.0.1:46916 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 07:14:30.817032 18138 ts_tablet_manager.cc:1905] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:30.818437 18138 ts_tablet_manager.cc:1918] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:30.818508 18138 log.cc:1196] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-0/wal/wals/d2403bedfe4c40e09c44ac6ce85d4742
I20260924 07:14:30.818758 18138 ts_tablet_manager.cc:1939] T d2403bedfe4c40e09c44ac6ce85d4742 P 779bea9f583b47c09308fb3a22d42447: Deleting consensus metadata
W20260924 07:14:30.819053 17702 connection.cc:441] server connection from 127.0.0.1:38448 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38448 (request call id 1) could send its response
I20260924 07:14:30.824355 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.817890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40722 (local address 127.10.130.253:34245)
0924 07:14:30.817927 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.817929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.818076 (+ 147us) server_negotiation.cc:407] Connection header received
0924 07:14:30.818164 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.818165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.818190 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.818219 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.818910 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.819610 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.820432 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.820640 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.820693 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.822166 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.822168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.822168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.822174 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.823400 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.823684 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.823685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.823686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.823732 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.823892 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.823893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.823893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.824013 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.824101 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.824192 (+ 91us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.824263 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:14:30.825237 18099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:30.817980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.10.130.252:38911)
0924 07:14:30.818015 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:30.818017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:30.818126 (+ 109us) server_negotiation.cc:407] Connection header received
0924 07:14:30.818186 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:30.818186 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:30.818198 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:30.818241 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:30.818796 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.819551 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.820772 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:30.821023 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:30.821083 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:30.822209 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:30.822210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:30.822210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:30.822218 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:30.824160 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.824603 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.824604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.824604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.824653 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:30.824831 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:30.824832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:30.824832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:30.824935 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:30.825027 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:30.825120 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 07:14:30.825180 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:14:31.805862 18147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:31.805378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.193:36631)
0924 07:14:31.805727 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:31.805807 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":78,"threads_started":1}
I20260924 07:14:31.806051 18146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:31.805373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.194:51609)
0924 07:14:31.805868 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:31.806003 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":63,"threads_started":1}
I20260924 07:14:31.806275 18148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:31.806025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.195:33061)
0924 07:14:31.806172 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:31.806235 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":45,"threads_started":1}
W20260924 07:14:31.806272 17815 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:40779 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
W20260924 07:14:31.806281 17951 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:40779 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
W20260924 07:14:31.806453 18087 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:40779 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
I20260924 07:14:32.308529 18149 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 433bb3767a8c44619896a663d6fcdf67)
I20260924 07:14:32.308647 18149 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } }
I20260924 07:14:32.308851 18150 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 433bb3767a8c44619896a663d6fcdf67)
I20260924 07:14:32.308976 18150 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } }
I20260924 07:14:32.309207 18150 leader_election.cc:288] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 433bb3767a8c44619896a663d6fcdf67 (127.10.130.254:40779), a7e7537f3da642b095ac9def4634cb86 (127.10.130.252:38911)
I20260924 07:14:32.309264 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.309065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:36426)
0924 07:14:32.309160 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.309220 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":46,"threads_started":1}
W20260924 07:14:32.309468 17620 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
I20260924 07:14:32.309602 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.309319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:36440)
0924 07:14:32.309514 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.309565 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":44,"threads_started":1}
I20260924 07:14:32.309674 18149 leader_election.cc:288] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 433bb3767a8c44619896a663d6fcdf67 (127.10.130.254:40779), 5b151bdf2b7f4959b1d7bcb788a3c687 (127.10.130.253:34245)
W20260924 07:14:32.309823 17434 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
I20260924 07:14:32.309842 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.309765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:36452)
0924 07:14:32.309797 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.309817 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260924 07:14:32.309911 17620 leader_election.cc:334] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 433bb3767a8c44619896a663d6fcdf67 (127.10.130.254:40779): Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
I20260924 07:14:32.310252 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.310171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:36456)
0924 07:14:32.310207 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.310227 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 07:14:32.310325 17434 leader_election.cc:334] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 433bb3767a8c44619896a663d6fcdf67 (127.10.130.254:40779): Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
I20260924 07:14:32.311868 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.309208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40734 (local address 127.10.130.253:34245)
0924 07:14:32.309324 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:32.309326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:32.309333 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:32.309356 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:32.309357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:32.309367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:32.309419 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:32.309568 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.310277 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.311233 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.311596 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.311652 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.311661 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:32.311777 (+ 116us) server_negotiation.cc:299] Negotiation successful
0924 07:14:32.311809 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":38,"threads_started":1}
I20260924 07:14:32.311879 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.309082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:34245 (local address 127.0.0.1:40734)
0924 07:14:32.309167 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.309177 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:32.309229 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:32.309435 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:32.309436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:32.309438 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:32.309494 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.309496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.310386 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.310387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.311154 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.311157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.311666 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.311667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.311718 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.311722 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:32.311767 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 07:14:32.311807 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"thread_start_us":43,"threads_started":1}
I20260924 07:14:32.312141 17457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7e7537f3da642b095ac9def4634cb86" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" is_pre_election: true
I20260924 07:14:32.312243 17457 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7e7537f3da642b095ac9def4634cb86 in term 1.
I20260924 07:14:32.312322 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.309442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38911 (local address 127.0.0.1:41990)
0924 07:14:32.309783 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.309793 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:32.309851 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:32.310163 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:32.310164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:32.310165 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:32.310209 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.310211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.310832 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.310833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.311568 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.311570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.312111 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.312112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.312175 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.312179 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:32.312218 (+ 39us) client_negotiation.cc:240] Negotiation successful
0924 07:14:32.312257 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":33,"threads_started":1}
I20260924 07:14:32.312336 18157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.309580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.10.130.252:38911)
0924 07:14:32.310046 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:32.310048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:32.310057 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:32.310077 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:32.310078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:32.310087 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:32.310152 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:32.310259 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.310765 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.311640 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.312052 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.312110 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.312117 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:32.312235 (+ 118us) server_negotiation.cc:299] Negotiation successful
0924 07:14:32.312273 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":72,"threads_started":1}
I20260924 07:14:32.312400 17617 leader_election.cc:302] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b151bdf2b7f4959b1d7bcb788a3c687, a7e7537f3da642b095ac9def4634cb86; no voters: 433bb3767a8c44619896a663d6fcdf67
I20260924 07:14:32.312503 18149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 07:14:32.312543 18149 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 433bb3767a8c44619896a663d6fcdf67)
I20260924 07:14:32.312570 18149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 1 FOLLOWER]: Advancing to term 2
I20260924 07:14:32.312631 17642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a7e7537f3da642b095ac9def4634cb86" is_pre_election: true
I20260924 07:14:32.313277 18149 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } }
I20260924 07:14:32.313400 17642 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b151bdf2b7f4959b1d7bcb788a3c687 in current term 2: Already voted for candidate a7e7537f3da642b095ac9def4634cb86 in this term.
I20260924 07:14:32.313398 18149 leader_election.cc:288] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [CANDIDATE]: Term 2 election: Requested vote from peers 433bb3767a8c44619896a663d6fcdf67 (127.10.130.254:40779), 5b151bdf2b7f4959b1d7bcb788a3c687 (127.10.130.253:34245)
I20260924 07:14:32.313544 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.313463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:36464)
0924 07:14:32.313497 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.313519 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260924 07:14:32.313527 17457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7e7537f3da642b095ac9def4634cb86" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687"
I20260924 07:14:32.313587 17457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 1 FOLLOWER]: Advancing to term 2
I20260924 07:14:32.313601 17434 leader_election.cc:302] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b151bdf2b7f4959b1d7bcb788a3c687; no voters: 433bb3767a8c44619896a663d6fcdf67, a7e7537f3da642b095ac9def4634cb86
I20260924 07:14:32.313797 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.313718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:36468)
0924 07:14:32.313757 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.313777 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 07:14:32.313867 17620 leader_election.cc:334] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 433bb3767a8c44619896a663d6fcdf67 (127.10.130.254:40779): Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
I20260924 07:14:32.314180 17457 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7e7537f3da642b095ac9def4634cb86 in term 2.
I20260924 07:14:32.314237 18150 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 07:14:32.314337 17617 leader_election.cc:302] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b151bdf2b7f4959b1d7bcb788a3c687, a7e7537f3da642b095ac9def4634cb86; no voters: 433bb3767a8c44619896a663d6fcdf67
I20260924 07:14:32.314414 18149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 07:14:32.314540 18149 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [term 2 LEADER]: Becoming Leader. State: Replica: a7e7537f3da642b095ac9def4634cb86, State: Running, Role: LEADER
I20260924 07:14:32.314604 18149 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } }
I20260924 07:14:32.314909 18159 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7e7537f3da642b095ac9def4634cb86. Latest consensus state: current_term: 2 leader_uuid: "a7e7537f3da642b095ac9def4634cb86" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } } }
I20260924 07:14:32.314980 18159 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:32.315111 18161 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:32.315466 18161 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:32.315529 18161 catalog_manager.cc:1259] Loaded cluster ID: e06d2fbb994c47c180c5145efd658cc4
I20260924 07:14:32.315559 18161 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:32.315704 18161 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:32.315771 18161 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: Loaded TSK: 0
I20260924 07:14:32.315899 18161 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:14:32.350962 17631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41988:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:14:32.352725 17631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:32.353627 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.353536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:36484)
0924 07:14:32.353568 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.353599 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260924 07:14:32.353657 17457 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [term 2 FOLLOWER]: Refusing update from remote peer a7e7537f3da642b095ac9def4634cb86: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260924 07:14:32.353962 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.353837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.0.0.1:36488)
0924 07:14:32.353880 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.353908 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260924 07:14:32.353912 18159 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260924 07:14:32.354036 17620 consensus_peers.cc:487] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 -> Peer 433bb3767a8c44619896a663d6fcdf67 (127.10.130.254:40779): Couldn't send request to peer 433bb3767a8c44619896a663d6fcdf67. Status: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 07:14:32.354462 18150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7e7537f3da642b095ac9def4634cb86. Latest consensus state: current_term: 2 leader_uuid: "a7e7537f3da642b095ac9def4634cb86" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } } }
I20260924 07:14:32.354543 18150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:14:32.354836 18162 mvcc.cc:205] Tried to move back new op lower bound from 7332798760358735872 to 7332798760201064448. Current Snapshot: MvccSnapshot[applied={T|T < 7332798760358735872}]
I20260924 07:14:32.354955 18159 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a7e7537f3da642b095ac9def4634cb86" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } } }
I20260924 07:14:32.355031 18159 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:32.355235 18150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a7e7537f3da642b095ac9def4634cb86" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "433bb3767a8c44619896a663d6fcdf67" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40779 } } peers { permanent_uuid: "5b151bdf2b7f4959b1d7bcb788a3c687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 34245 } attrs { promote: false } } peers { permanent_uuid: "a7e7537f3da642b095ac9def4634cb86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38911 } attrs { promote: false } } }
I20260924 07:14:32.355326 18150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b151bdf2b7f4959b1d7bcb788a3c687 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:14:32.359318 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.356518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38491 (local address 127.0.0.1:43380)
0924 07:14:32.356552 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.356562 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:32.356595 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:32.356933 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:32.356934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:32.356936 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:32.356983 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.356985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.357780 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.357781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.358544 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.358546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.359125 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.359126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.359184 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.359189 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 07:14:32.359218 (+ 29us) client_negotiation.cc:240] Negotiation successful
0924 07:14:32.359253 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":1}
I20260924 07:14:32.359568 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.356672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43380 (local address 127.10.130.195:38491)
0924 07:14:32.356793 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:32.356795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:32.356804 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:32.356829 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:32.356829 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:32.356840 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:32.357048 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:32.357086 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.357704 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.358615 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.359037 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.359424 (+ 387us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.359435 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:32.359453 (+ 18us) server_negotiation.cc:299] Negotiation successful
0924 07:14:32.359494 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":55,"threads_started":1}
I20260924 07:14:32.359817 18020 tablet_service.cc:1467] Processing CreateTablet for tablet 5838c9239ded4122b2d1e5fb4711db75 (DEFAULT_TABLE table=test-table [id=53c4f38b5c934d67ad5ac4a27dc37a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:32.359858 18151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.356563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:41189 (local address 127.0.0.1:38470)
0924 07:14:32.356592 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.356601 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:32.356654 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:32.357002 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:32.357003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:32.357006 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:32.357051 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.357052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.357854 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.357855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.358942 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.358944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.359625 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.359627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.359698 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.359702 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:32.359752 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 07:14:32.359789 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260924 07:14:32.359885 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.356743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38470 (local address 127.10.130.193:41189)
0924 07:14:32.356847 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:32.356848 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:32.356855 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:32.356879 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:32.356880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:32.356891 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:32.357111 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:32.357150 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.357760 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.359037 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.359531 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.359586 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.359595 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:32.359770 (+ 175us) server_negotiation.cc:299] Negotiation successful
0924 07:14:32.359819 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":62,"threads_started":1}
I20260924 07:14:32.359946 18020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5838c9239ded4122b2d1e5fb4711db75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:32.360121 17748 tablet_service.cc:1467] Processing CreateTablet for tablet 5838c9239ded4122b2d1e5fb4711db75 (DEFAULT_TABLE table=test-table [id=53c4f38b5c934d67ad5ac4a27dc37a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:32.360203 17748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5838c9239ded4122b2d1e5fb4711db75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:32.360723 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.356645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:38631 (local address 127.0.0.1:46930)
0924 07:14:32.356995 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.357004 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:32.357137 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:32.358032 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:32.358032 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:32.358034 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:32.358067 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.358068 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.358933 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.358933 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.360008 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.360011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.360513 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.360514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.360572 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.360576 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:32.360619 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 07:14:32.360653 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":305,"threads_started":1}
I20260924 07:14:32.360738 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.356911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46930 (local address 127.10.130.194:38631)
0924 07:14:32.357898 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:32.357900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:32.357907 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:32.357929 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:32.357930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:32.357940 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:32.357992 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:32.358129 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.358862 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.360066 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.360447 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.360506 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.360514 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:32.360629 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 07:14:32.360668 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":967,"thread_start_us":53,"threads_started":1}
I20260924 07:14:32.360963 17884 tablet_service.cc:1467] Processing CreateTablet for tablet 5838c9239ded4122b2d1e5fb4711db75 (DEFAULT_TABLE table=test-table [id=53c4f38b5c934d67ad5ac4a27dc37a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:32.361070 17884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5838c9239ded4122b2d1e5fb4711db75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:32.361325 18172 tablet_bootstrap.cc:492] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447: Bootstrap starting.
I20260924 07:14:32.361763 18172 tablet_bootstrap.cc:654] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:32.362259 18172 tablet_bootstrap.cc:492] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447: No bootstrap required, opened a new log
I20260924 07:14:32.362303 18172 ts_tablet_manager.cc:1397] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:32.362399 18172 raft_consensus.cc:348] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } } peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } }
I20260924 07:14:32.362466 18172 raft_consensus.cc:374] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:32.362484 18172 raft_consensus.cc:729] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 779bea9f583b47c09308fb3a22d42447, State: Initialized, Role: FOLLOWER
I20260924 07:14:32.362527 18172 consensus_queue.cc:260] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } } peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } }
I20260924 07:14:32.362655 18172 ts_tablet_manager.cc:1428] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:32.363173 17814 heartbeater.cc:503] Master 127.10.130.252:38911 was elected leader, sending a full tablet report...
I20260924 07:14:32.363436 18176 tablet_bootstrap.cc:492] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8: Bootstrap starting.
I20260924 07:14:32.363623 18174 tablet_bootstrap.cc:492] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e: Bootstrap starting.
I20260924 07:14:32.363641 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.363453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.193:60349)
0924 07:14:32.363564 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.363610 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":70,"threads_started":1}
I20260924 07:14:32.363794 18176 tablet_bootstrap.cc:654] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:32.364075 18174 tablet_bootstrap.cc:654] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:32.364310 18176 tablet_bootstrap.cc:492] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8: No bootstrap required, opened a new log
I20260924 07:14:32.364390 18176 ts_tablet_manager.cc:1397] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:32.364514 18176 raft_consensus.cc:348] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } } peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } }
I20260924 07:14:32.364567 18176 raft_consensus.cc:374] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:32.364580 18176 raft_consensus.cc:729] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e25329d24dfa425f8984014b7813fea8, State: Initialized, Role: FOLLOWER
I20260924 07:14:32.364598 18174 tablet_bootstrap.cc:492] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e: No bootstrap required, opened a new log
I20260924 07:14:32.364629 18176 consensus_queue.cc:260] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } } peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } }
I20260924 07:14:32.364648 18174 ts_tablet_manager.cc:1397] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 07:14:32.364765 18176 ts_tablet_manager.cc:1428] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:32.364766 18174 raft_consensus.cc:348] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } } peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } }
I20260924 07:14:32.364857 18174 raft_consensus.cc:374] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:32.364886 18174 raft_consensus.cc:729] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdc83d87f36a4efeb4076c1e3c68917e, State: Initialized, Role: FOLLOWER
I20260924 07:14:32.364933 18174 consensus_queue.cc:260] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } } peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } }
I20260924 07:14:32.365090 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.364908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.195:55277)
0924 07:14:32.365012 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.365048 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":65,"threads_started":1}
I20260924 07:14:32.365295 18086 heartbeater.cc:503] Master 127.10.130.252:38911 was elected leader, sending a full tablet report...
I20260924 07:14:32.365150 18174 ts_tablet_manager.cc:1428] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:32.365784 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.365488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.195:44493)
0924 07:14:32.365733 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.365759 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231}
I20260924 07:14:32.365854 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.365679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.194:38073)
0924 07:14:32.365789 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.365823 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":79,"threads_started":1}
I20260924 07:14:32.365859 17950 heartbeater.cc:503] Master 127.10.130.252:38911 was elected leader, sending a full tablet report...
I20260924 07:14:32.366238 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.366101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.194:41357)
0924 07:14:32.366131 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.366216 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260924 07:14:32.390921 18180 raft_consensus.cc:482] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:32.391002 18180 raft_consensus.cc:504] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } } peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } }
I20260924 07:14:32.391322 18180 leader_election.cc:288] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 779bea9f583b47c09308fb3a22d42447 (127.10.130.193:41189), bdc83d87f36a4efeb4076c1e3c68917e (127.10.130.194:38631)
I20260924 07:14:32.394598 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.391621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:42617 (local address 127.10.130.194:38631)
0924 07:14:32.391689 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:32.391691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:32.391758 (+ 67us) server_negotiation.cc:407] Connection header received
0924 07:14:32.391819 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:32.391820 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:32.391828 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:32.391870 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:32.392000 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.392620 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.393905 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.394259 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.394319 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.394328 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:32.394490 (+ 162us) server_negotiation.cc:299] Negotiation successful
0924 07:14:32.394530 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 07:14:32.394748 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.391524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:33431 (local address 127.10.130.193:41189)
0924 07:14:32.391564 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:32.391567 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:32.391579 (+ 12us) server_negotiation.cc:407] Connection header received
0924 07:14:32.391608 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:32.391609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:32.391617 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:32.391669 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:32.391818 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.392380 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.393815 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.394342 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.394413 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.394424 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:32.394639 (+ 215us) server_negotiation.cc:299] Negotiation successful
0924 07:14:32.394682 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:14:32.394755 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.391462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:41189 (local address 127.10.130.195:33431)
0924 07:14:32.391501 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.391512 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:32.391559 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:32.391689 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:32.391690 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:32.391693 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:32.391764 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.391766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.392463 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.392464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.393685 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.393688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.394476 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.394478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.394560 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.394564 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:32.394604 (+ 40us) client_negotiation.cc:240] Negotiation successful
0924 07:14:32.394666 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 07:14:32.394876 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.391555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:38631 (local address 127.10.130.195:42617)
0924 07:14:32.391711 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.391719 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:32.391766 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:32.391880 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:32.391880 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:32.391882 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:32.391933 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.391935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.392707 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.392708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.393824 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:32.393826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.394325 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.394334 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:32.394405 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.394409 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:32.394776 (+ 367us) client_negotiation.cc:240] Negotiation successful
0924 07:14:32.394812 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260924 07:14:32.395052 17904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5838c9239ded4122b2d1e5fb4711db75" candidate_uuid: "e25329d24dfa425f8984014b7813fea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" is_pre_election: true
I20260924 07:14:32.395128 17904 raft_consensus.cc:2454] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e25329d24dfa425f8984014b7813fea8 in term 0.
I20260924 07:14:32.395205 17768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5838c9239ded4122b2d1e5fb4711db75" candidate_uuid: "e25329d24dfa425f8984014b7813fea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "779bea9f583b47c09308fb3a22d42447" is_pre_election: true
I20260924 07:14:32.395316 17768 raft_consensus.cc:2454] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e25329d24dfa425f8984014b7813fea8 in term 0.
I20260924 07:14:32.395318 17975 leader_election.cc:302] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdc83d87f36a4efeb4076c1e3c68917e, e25329d24dfa425f8984014b7813fea8; no voters:
I20260924 07:14:32.395573 18180 raft_consensus.cc:2781] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:32.395617 18180 raft_consensus.cc:482] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:32.395653 18180 raft_consensus.cc:3037] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:32.396323 18180 raft_consensus.cc:504] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } } peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } }
I20260924 07:14:32.396428 18180 leader_election.cc:288] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [CANDIDATE]: Term 1 election: Requested vote from peers 779bea9f583b47c09308fb3a22d42447 (127.10.130.193:41189), bdc83d87f36a4efeb4076c1e3c68917e (127.10.130.194:38631)
I20260924 07:14:32.396565 17768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5838c9239ded4122b2d1e5fb4711db75" candidate_uuid: "e25329d24dfa425f8984014b7813fea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "779bea9f583b47c09308fb3a22d42447"
I20260924 07:14:32.396602 17904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5838c9239ded4122b2d1e5fb4711db75" candidate_uuid: "e25329d24dfa425f8984014b7813fea8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdc83d87f36a4efeb4076c1e3c68917e"
I20260924 07:14:32.396644 17768 raft_consensus.cc:3037] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:32.396658 17904 raft_consensus.cc:3037] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:32.397310 17904 raft_consensus.cc:2454] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e25329d24dfa425f8984014b7813fea8 in term 1.
I20260924 07:14:32.397383 17768 raft_consensus.cc:2454] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e25329d24dfa425f8984014b7813fea8 in term 1.
I20260924 07:14:32.397483 17975 leader_election.cc:302] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdc83d87f36a4efeb4076c1e3c68917e, e25329d24dfa425f8984014b7813fea8; no voters:
I20260924 07:14:32.397576 18180 raft_consensus.cc:2781] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:32.397701 18180 raft_consensus.cc:686] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [term 1 LEADER]: Becoming Leader. State: Replica: e25329d24dfa425f8984014b7813fea8, State: Running, Role: LEADER
I20260924 07:14:32.397758 18180 consensus_queue.cc:237] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } } peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } }
I20260924 07:14:32.397972 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.397880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.195:42333)
0924 07:14:32.397913 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.397945 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260924 07:14:32.398416 17633 catalog_manager.cc:5495] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e25329d24dfa425f8984014b7813fea8 (127.10.130.195). New cstate: current_term: 1 leader_uuid: "e25329d24dfa425f8984014b7813fea8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e25329d24dfa425f8984014b7813fea8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 } health_report { overall_health: UNKNOWN } } }
I20260924 07:14:32.404215 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.402266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43382 (local address 127.10.130.195:38491)
0924 07:14:32.402298 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:32.402300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:32.402329 (+ 29us) server_negotiation.cc:407] Connection header received
0924 07:14:32.402356 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:32.402357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:32.402372 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:32.402419 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:32.402498 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.402932 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.403595 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:32.403784 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:32.403841 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:32.403934 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:32.404066 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:32.404120 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:32.404149 (+ 29us) server_negotiation.cc:299] Negotiation successful
0924 07:14:32.404165 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:14:32.405073 17904 raft_consensus.cc:1261] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e [term 1 FOLLOWER]: Refusing update from remote peer e25329d24dfa425f8984014b7813fea8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:32.405148 17768 raft_consensus.cc:1261] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447 [term 1 FOLLOWER]: Refusing update from remote peer e25329d24dfa425f8984014b7813fea8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:32.405344 18185 consensus_queue.cc:1035] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdc83d87f36a4efeb4076c1e3c68917e" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:32.405442 18180 consensus_queue.cc:1035] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "779bea9f583b47c09308fb3a22d42447" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:32.406198 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.406100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.194:49951)
0924 07:14:32.406137 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.406170 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 07:14:32.406484 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.406390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.193:44297)
0924 07:14:32.406427 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.406457 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 07:14:32.407243 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.407135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.195:57931)
0924 07:14:32.407186 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.407215 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 07:14:32.407634 18183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.407532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.194:41669)
0924 07:14:32.407581 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.407608 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 07:14:32.407898 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.407825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.195:45781)
0924 07:14:32.407854 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.407875 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 07:14:32.408219 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:32.408086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40779 (local address 127.10.130.193:41289)
0924 07:14:32.408159 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.408195 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:40779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260924 07:14:32.408459 18191 mvcc.cc:205] Tried to move back new op lower bound from 7332798760569540608 to 7332798760541720576. Current Snapshot: MvccSnapshot[applied={T|T < 7332798760569540608 or (T in {7332798760569540608})}]
I20260924 07:14:32.409698 17633 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:32.409766 17633 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41988:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:32.410880 17633 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a7e7537f3da642b095ac9def4634cb86: Sending DeleteTablet for 3 replicas of tablet 5838c9239ded4122b2d1e5fb4711db75
I20260924 07:14:32.411127 17748 tablet_service.cc:1514] Processing DeleteTablet for tablet 5838c9239ded4122b2d1e5fb4711db75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:32 UTC) from {username='oryx', principal='oryx/127.10.130.252@KRBTEST.COM'} at 127.0.0.1:38470
I20260924 07:14:32.411163 17884 tablet_service.cc:1514] Processing DeleteTablet for tablet 5838c9239ded4122b2d1e5fb4711db75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:32 UTC) from {username='oryx', principal='oryx/127.10.130.252@KRBTEST.COM'} at 127.0.0.1:46930
I20260924 07:14:32.411209 18020 tablet_service.cc:1514] Processing DeleteTablet for tablet 5838c9239ded4122b2d1e5fb4711db75 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:32 UTC) from {username='oryx', principal='oryx/127.10.130.252@KRBTEST.COM'} at 127.0.0.1:43380
I20260924 07:14:32.411296 18197 tablet_replica.cc:330] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447: stopping tablet replica
I20260924 07:14:32.411339 18197 raft_consensus.cc:2229] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:32.411371 18197 raft_consensus.cc:2258] T 5838c9239ded4122b2d1e5fb4711db75 P 779bea9f583b47c09308fb3a22d42447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:32.411499 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 17685
I20260924 07:14:32.412055 18198 tablet_replica.cc:330] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e: stopping tablet replica
I20260924 07:14:32.412103 18198 raft_consensus.cc:2229] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:32.412137 18198 raft_consensus.cc:2258] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:32.412236 18199 tablet_replica.cc:330] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8: stopping tablet replica
I20260924 07:14:32.412281 18199 raft_consensus.cc:2229] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:32.412340 18199 raft_consensus.cc:2258] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:32.412542 18199 ts_tablet_manager.cc:1905] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:32.414000 18199 ts_tablet_manager.cc:1918] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:32.414054 18199 log.cc:1196] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/5838c9239ded4122b2d1e5fb4711db75
I20260924 07:14:32.414260 18199 ts_tablet_manager.cc:1939] T 5838c9239ded4122b2d1e5fb4711db75 P e25329d24dfa425f8984014b7813fea8: Deleting consensus metadata
I20260924 07:14:32.414549 17617 catalog_manager.cc:4841] TS e25329d24dfa425f8984014b7813fea8 (127.10.130.195:38491): tablet 5838c9239ded4122b2d1e5fb4711db75 (table test-table [id=53c4f38b5c934d67ad5ac4a27dc37a5b]) successfully deleted
I20260924 07:14:32.416213 18198 ts_tablet_manager.cc:1905] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:32.417667 18198 ts_tablet_manager.cc:1918] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:32.417760 18198 log.cc:1196] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/5838c9239ded4122b2d1e5fb4711db75
I20260924 07:14:32.418012 18198 ts_tablet_manager.cc:1939] T 5838c9239ded4122b2d1e5fb4711db75 P bdc83d87f36a4efeb4076c1e3c68917e: Deleting consensus metadata
I20260924 07:14:32.418329 17619 catalog_manager.cc:4841] TS bdc83d87f36a4efeb4076c1e3c68917e (127.10.130.194:38631): tablet 5838c9239ded4122b2d1e5fb4711db75 (table test-table [id=53c4f38b5c934d67ad5ac4a27dc37a5b]) successfully deleted
W20260924 07:14:32.418627 18170 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:32.418281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:41189 (local address 127.0.0.1:38482)
0924 07:14:32.418350 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:32.418361 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:32.418394 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:32.418594 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:41189: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":55}
W20260924 07:14:32.418745 17619 catalog_manager.cc:4584] TS 779bea9f583b47c09308fb3a22d42447 (127.10.130.193:41189): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5838c9239ded4122b2d1e5fb4711db75: Network error: Client connection negotiation failed: client connection to 127.10.130.193:41189: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 07:14:32.418977 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 17821
I20260924 07:14:32.425104 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 17957
I20260924 07:14:32.429788 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 17314
I20260924 07:14:32.433745 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 17501
2026-09-24T07:14:32Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4227 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:32 dist-test-slave-7q7m krb5kdc[18202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:32 dist-test-slave-7q7m krb5kdc[18202](info): set up 2 sockets
Sep 24 07:14:32 dist-test-slave-7q7m krb5kdc[18202](info): commencing operation
krb5kdc: starting...
W20260924 07:14:33.375999 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:33 dist-test-slave-7q7m krb5kdc[18202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:33Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.254.keytab.
Entry for principal oryx/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.254.keytab.
I20260924 07:14:33.480561 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:35053
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.254.keytab
--principal=oryx/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:38111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:35053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:33.557476 18215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:33.557660 18215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:33.557680 18215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:33.558738 18215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:33.558769 18215 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 07:14:33.558780 18215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:33.558792 18215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:33.558816 18215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:33.558827 18215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:33.560325 18215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:35053
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.254.keytab
--principal=oryx/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:35053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:33.560508 18215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:33.560665 18215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:33.564980 18221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:33.564998 18215 server_base.cc:1034] running on GCE node
W20260924 07:14:33.564960 18223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:33.564970 18220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:33.565485 18215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:33.565723 18215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:33.566856 18215 hybrid_clock.cc:648] HybridClock initialized: now 1790234073566848 us; error 32 us; skew 500 ppm
Sep 24 07:14:33 dist-test-slave-7q7m krb5kdc[18202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234073, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:33.571936 18215 init.cc:377] Logged in from keytab as oryx/127.10.130.254@KRBTEST.COM (short username oryx)
I20260924 07:14:33.572333 18215 webserver.cc:460] Webserver started at http://127.10.130.254:44967/ using document root <none> and password file <none>
I20260924 07:14:33.572518 18215 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:33.572556 18215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:33.572641 18215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:33.573609 18215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "9ad253ca82294b5fae64f46deb86ce1c"
format_stamp: "Formatted at 2026-09-24 07:14:33 on dist-test-slave-7q7m"
server_key: "541fa2dfa4c4e29ac2ba1165ba73f8b8"
server_key_iv: "741e0c5b862d724e1e993b911f3e3653"
server_key_version: "encryptionkey@0"
I20260924 07:14:33.573890 18215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "9ad253ca82294b5fae64f46deb86ce1c"
format_stamp: "Formatted at 2026-09-24 07:14:33 on dist-test-slave-7q7m"
server_key: "541fa2dfa4c4e29ac2ba1165ba73f8b8"
server_key_iv: "741e0c5b862d724e1e993b911f3e3653"
server_key_version: "encryptionkey@0"
I20260924 07:14:33.575047 18215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:14:33.575651 18230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:33.575780 18215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:33.575853 18215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "9ad253ca82294b5fae64f46deb86ce1c"
format_stamp: "Formatted at 2026-09-24 07:14:33 on dist-test-slave-7q7m"
server_key: "541fa2dfa4c4e29ac2ba1165ba73f8b8"
server_key_iv: "741e0c5b862d724e1e993b911f3e3653"
server_key_version: "encryptionkey@0"
I20260924 07:14:33.575915 18215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:33.600662 18215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:33.602218 18215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:33.602319 18215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:33.606362 18215 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:35053
I20260924 07:14:33.606407 18292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:35053 every 8 connection(s)
I20260924 07:14:33.606696 18215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:33.607290 18293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:33.609491 18293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c: Bootstrap starting.
I20260924 07:14:33.610026 18293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:33.610275 18293 log.cc:824] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:33.610870 18293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c: No bootstrap required, opened a new log
I20260924 07:14:33.612819 18293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad253ca82294b5fae64f46deb86ce1c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 35053 } }
I20260924 07:14:33.612946 18293 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:33.612977 18293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ad253ca82294b5fae64f46deb86ce1c, State: Initialized, Role: FOLLOWER
I20260924 07:14:33.613107 18293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad253ca82294b5fae64f46deb86ce1c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 35053 } }
I20260924 07:14:33.613179 18293 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:33.613211 18293 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:33.613256 18293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:33.613708 18293 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad253ca82294b5fae64f46deb86ce1c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 35053 } }
I20260924 07:14:33.613790 18293 leader_election.cc:302] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ad253ca82294b5fae64f46deb86ce1c; no voters:
I20260924 07:14:33.613931 18293 leader_election.cc:288] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:33.613969 18296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:33.614104 18296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [term 1 LEADER]: Becoming Leader. State: Replica: 9ad253ca82294b5fae64f46deb86ce1c, State: Running, Role: LEADER
I20260924 07:14:33.614135 18293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:33.614187 18296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad253ca82294b5fae64f46deb86ce1c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 35053 } }
I20260924 07:14:33.614483 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 18215
I20260924 07:14:33.614529 18297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ad253ca82294b5fae64f46deb86ce1c. Latest consensus state: current_term: 1 leader_uuid: "9ad253ca82294b5fae64f46deb86ce1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad253ca82294b5fae64f46deb86ce1c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 35053 } } }
I20260924 07:14:33.614521 18298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ad253ca82294b5fae64f46deb86ce1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ad253ca82294b5fae64f46deb86ce1c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 35053 } } }
I20260924 07:14:33.614588 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:33.614605 18297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:33.614609 18298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:33.614791 10763 external_mini_cluster.cc:1428] Setting key 7e3588f58eeec8b0e8903b4f9059d292
I20260924 07:14:33.615322 18304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:33.615769 18304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:33.617669 18304 catalog_manager.cc:1347] Generated new cluster ID: 998bec77fc8d4124b2e8e4bc219684eb
I20260924 07:14:33.617712 18304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:14:33 dist-test-slave-7q7m krb5kdc[18202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234073, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
I20260924 07:14:33.627049 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:33.616318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47908 (local address 127.10.130.254:35053)
0924 07:14:33.616469 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:33.616477 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:33.616508 (+ 31us) server_negotiation.cc:407] Connection header received
0924 07:14:33.616614 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:33.616621 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:33.616683 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:33.616824 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:33.617403 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.618796 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.619896 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.620529 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.620954 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:33.623324 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:33.623328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:33.623328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:33.623339 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:33.625128 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:33.625510 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:33.625513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:33.625513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:33.625610 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:33.625815 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:33.625816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:33.625816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:33.626106 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:33.626621 (+ 515us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:33.626691 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 07:14:33.626833 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260924 07:14:33.628247 18304 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:33.628939 18304 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:33.639444 18304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c: Generated new TSK 0
I20260924 07:14:33.639592 18304 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab.
Entry for principal oryx/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab.
I20260924 07:14:33.664434 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:35053
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab
--principal=oryx/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:38111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:33.741328 18319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:33.741510 18319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:33.741540 18319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:33.742640 18319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:33.742676 18319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:33.742702 18319 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 07:14:33.742726 18319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:33.744220 18319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab
--principal=oryx/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.130.254:35053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:33.744436 18319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:33.744632 18319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:33.749148 18327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:33.749152 18324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:33.749137 18325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:33.749289 18319 server_base.cc:1034] running on GCE node
I20260924 07:14:33.749527 18319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:33.749702 18319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:33.750836 18319 hybrid_clock.cc:648] HybridClock initialized: now 1790234073750825 us; error 28 us; skew 500 ppm
Sep 24 07:14:33 dist-test-slave-7q7m krb5kdc[18202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234073, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:33.753062 18319 init.cc:377] Logged in from keytab as oryx/127.10.130.193@KRBTEST.COM (short username oryx)
I20260924 07:14:33.753429 18319 webserver.cc:460] Webserver started at http://127.10.130.193:42415/ using document root <none> and password file <none>
I20260924 07:14:33.753616 18319 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:33.753662 18319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:33.753763 18319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:33.754667 18319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "6e1f2b0ba4aa42298262543d325ea10b"
format_stamp: "Formatted at 2026-09-24 07:14:33 on dist-test-slave-7q7m"
server_key: "70b8036b84dd6d4b2d4ed0258172fd63"
server_key_iv: "fd176c2aa8ed459e57bdb6a43c6ed317"
server_key_version: "encryptionkey@0"
I20260924 07:14:33.754954 18319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "6e1f2b0ba4aa42298262543d325ea10b"
format_stamp: "Formatted at 2026-09-24 07:14:33 on dist-test-slave-7q7m"
server_key: "70b8036b84dd6d4b2d4ed0258172fd63"
server_key_iv: "fd176c2aa8ed459e57bdb6a43c6ed317"
server_key_version: "encryptionkey@0"
I20260924 07:14:33.756088 18319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:14:33.756703 18334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:33.756841 18319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:33.756913 18319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "6e1f2b0ba4aa42298262543d325ea10b"
format_stamp: "Formatted at 2026-09-24 07:14:33 on dist-test-slave-7q7m"
server_key: "70b8036b84dd6d4b2d4ed0258172fd63"
server_key_iv: "fd176c2aa8ed459e57bdb6a43c6ed317"
server_key_version: "encryptionkey@0"
I20260924 07:14:33.756970 18319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:33.775137 18319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:33.776628 18319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:33.776741 18319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:33.777129 18319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:33.777164 18319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:33.777189 18319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:33.777215 18319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:33.783033 18319 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:46589
I20260924 07:14:33.783093 18449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:46589 every 8 connection(s)
Sep 24 07:14:33 dist-test-slave-7q7m krb5kdc[18202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234073, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
I20260924 07:14:33.783443 18319 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:14:33.786047 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:33.777484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:35025 (local address 127.10.130.254:35053)
0924 07:14:33.777542 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:33.777545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:33.777701 (+ 156us) server_negotiation.cc:407] Connection header received
0924 07:14:33.778141 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:33.778142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:33.778153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:33.778251 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:33.779006 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.779762 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.781217 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.781523 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.781595 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:33.783840 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:33.783841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:33.783841 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:33.783851 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:33.785118 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:33.785423 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:33.785424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:33.785424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:33.785468 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:33.785617 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:33.785617 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:33.785617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:33.785704 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:33.785746 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:33.785933 (+ 187us) server_negotiation.cc:299] Negotiation successful
0924 07:14:33.785988 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 07:14:33.786202 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:33.777454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:35053 (local address 127.10.130.193:35025)
0924 07:14:33.777591 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:33.777626 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:33.778058 (+ 432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:33.778325 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:33.778329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:33.778661 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:33.778915 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:33.778921 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.779864 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.779866 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:33.781118 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:33.781121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.781596 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.781597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:33.781693 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:33.782148 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:33.782158 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:33.783740 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:33.785174 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:33.785178 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:33.785180 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:33.785369 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:33.785503 (+ 134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:33.785504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:33.785504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:33.785580 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:33.785744 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:33.785746 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:33.785848 (+ 102us) client_negotiation.cc:769] Sending connection context
0924 07:14:33.785920 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 07:14:33.786022 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
I20260924 07:14:33.786614 18450 heartbeater.cc:347] Connected to a master server at 127.10.130.254:35053
I20260924 07:14:33.786689 18450 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:33.786859 18450 heartbeater.cc:511] Master 127.10.130.254:35053 requested a full tablet report, sending...
I20260924 07:14:33.787362 18246 ts_manager.cc:194] Registered new tserver with Master: 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193:46589)
I20260924 07:14:33.788058 18246 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:35025
I20260924 07:14:33.788201 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 18319
I20260924 07:14:33.788314 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:33.788491 10763 external_mini_cluster.cc:1428] Setting key 5a922941aef747610764fa0fab58d749
I20260924 07:14:33.792740 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:33.787555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:58193 (local address 127.10.130.254:35053)
0924 07:14:33.787592 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:33.787594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:33.787684 (+ 90us) server_negotiation.cc:407] Connection header received
0924 07:14:33.787731 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:33.787732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:33.787740 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:33.787790 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:33.788136 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.788804 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.789544 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.789737 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.789787 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:33.790637 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:33.790639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:33.790639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:33.790644 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:33.791752 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:33.792109 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:33.792109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:33.792110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:33.792155 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:33.792330 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:33.792330 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:33.792330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:33.792415 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:33.792455 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:33.792614 (+ 159us) server_negotiation.cc:299] Negotiation successful
0924 07:14:33.792688 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:14:33.792788 18456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:33.787516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:35053 (local address 127.10.130.193:58193)
0924 07:14:33.787626 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:33.787635 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:33.787680 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:33.787794 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:33.787794 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:33.787960 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:33.788049 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:33.788051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.788918 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.788918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:33.789476 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:33.789478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.789790 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.789791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:33.789862 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:33.790224 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:33.790229 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:33.790552 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:33.791827 (+ 1275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:33.791828 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:33.791828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:33.792040 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:33.792210 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:33.792211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:33.792211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:33.792270 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:33.792461 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:33.792462 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:33.792503 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 07:14:33.792601 (+ 98us) client_negotiation.cc:240] Negotiation successful
0924 07:14:33.792690 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
WARNING: no policy specified for oryx/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab.
Entry for principal oryx/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab.
I20260924 07:14:33.808296 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:35053
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab
--principal=oryx/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:38111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:33.885054 18459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:33.885248 18459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:33.885285 18459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:33.886430 18459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:33.886464 18459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:33.886490 18459 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 07:14:33.886513 18459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:33.888000 18459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab
--principal=oryx/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.130.254:35053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:33.888219 18459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:33.888415 18459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:33.892887 18464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:33.892891 18465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:33.892903 18467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:33.893090 18459 server_base.cc:1034] running on GCE node
I20260924 07:14:33.893287 18459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:33.893509 18459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:33.894655 18459 hybrid_clock.cc:648] HybridClock initialized: now 1790234073894635 us; error 32 us; skew 500 ppm
Sep 24 07:14:33 dist-test-slave-7q7m krb5kdc[18202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234073, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:33.896916 18459 init.cc:377] Logged in from keytab as oryx/127.10.130.194@KRBTEST.COM (short username oryx)
I20260924 07:14:33.897318 18459 webserver.cc:460] Webserver started at http://127.10.130.194:34391/ using document root <none> and password file <none>
I20260924 07:14:33.897500 18459 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:33.897545 18459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:33.897658 18459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:33.898577 18459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "54ff4dd74c094d099b72951152543b70"
format_stamp: "Formatted at 2026-09-24 07:14:33 on dist-test-slave-7q7m"
server_key: "64a1a5c5f7d64414d082aa38c64bcea5"
server_key_iv: "911d5b8800a190d2b0d2fe29f96a4820"
server_key_version: "encryptionkey@0"
I20260924 07:14:33.898887 18459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "54ff4dd74c094d099b72951152543b70"
format_stamp: "Formatted at 2026-09-24 07:14:33 on dist-test-slave-7q7m"
server_key: "64a1a5c5f7d64414d082aa38c64bcea5"
server_key_iv: "911d5b8800a190d2b0d2fe29f96a4820"
server_key_version: "encryptionkey@0"
I20260924 07:14:33.900063 18459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:14:33.900671 18474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:33.900820 18459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:33.900897 18459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "54ff4dd74c094d099b72951152543b70"
format_stamp: "Formatted at 2026-09-24 07:14:33 on dist-test-slave-7q7m"
server_key: "64a1a5c5f7d64414d082aa38c64bcea5"
server_key_iv: "911d5b8800a190d2b0d2fe29f96a4820"
server_key_version: "encryptionkey@0"
I20260924 07:14:33.900959 18459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:33.944070 18459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:33.945736 18459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:33.945884 18459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:33.946363 18459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:33.946401 18459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:33.946430 18459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:33.946446 18459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 07:14:33 dist-test-slave-7q7m krb5kdc[18202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234073, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
I20260924 07:14:33.953003 18459 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:37809
I20260924 07:14:33.953087 18589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:37809 every 8 connection(s)
I20260924 07:14:33.953341 18459 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:14:33.955180 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:33.946788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:56965 (local address 127.10.130.254:35053)
0924 07:14:33.946830 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:33.946832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:33.947038 (+ 206us) server_negotiation.cc:407] Connection header received
0924 07:14:33.947564 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:33.947565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:33.947579 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:33.947639 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:33.948570 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.949054 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.950036 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.950224 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.950277 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:33.952565 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:33.952566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:33.952566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:33.952574 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:33.953986 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:33.954336 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:33.954337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:33.954337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:33.954400 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:33.954607 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:33.954608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:33.954609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:33.954716 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:33.954769 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:33.955020 (+ 251us) server_negotiation.cc:299] Negotiation successful
0924 07:14:33.955094 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 07:14:33.955327 18483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:33.946791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:35053 (local address 127.10.130.194:56965)
0924 07:14:33.946928 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:33.946961 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:33.947469 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:33.947756 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:33.947761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:33.948163 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:33.948462 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:33.948469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.949140 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.949141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:33.949946 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:33.949949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.950301 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.950303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:33.950418 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:33.950870 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:33.950881 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:33.952480 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:33.954053 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:33.954060 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:33.954062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:33.954272 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:33.954451 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:33.954451 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:33.954452 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:33.954554 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:33.954767 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:33.954773 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:33.954905 (+ 132us) client_negotiation.cc:769] Sending connection context
0924 07:14:33.954990 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 07:14:33.955133 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":77,"threads_started":1}
I20260924 07:14:33.955741 18590 heartbeater.cc:347] Connected to a master server at 127.10.130.254:35053
I20260924 07:14:33.955816 18590 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:33.955961 18590 heartbeater.cc:511] Master 127.10.130.254:35053 requested a full tablet report, sending...
I20260924 07:14:33.956326 18247 ts_manager.cc:194] Registered new tserver with Master: 54ff4dd74c094d099b72951152543b70 (127.10.130.194:37809)
I20260924 07:14:33.956779 18247 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:56965
I20260924 07:14:33.961138 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:33.956565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:35053 (local address 127.10.130.194:35989)
0924 07:14:33.956706 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:33.956714 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:33.956766 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:33.956912 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:33.956913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:33.957074 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:33.957134 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:33.957135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.957708 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.957709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:33.958200 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:33.958202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.958465 (+ 263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.958466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:33.958508 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:33.958864 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:33.958869 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:33.959110 (+ 241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:33.960429 (+ 1319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:33.960430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:33.960430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:33.960574 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:33.960711 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:33.960711 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:33.960711 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:33.960754 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:33.960938 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:33.960939 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:33.960970 (+ 31us) client_negotiation.cc:769] Sending connection context
0924 07:14:33.961014 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 07:14:33.961079 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":59,"threads_started":1}
I20260924 07:14:33.961138 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:33.956620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:35989 (local address 127.10.130.254:35053)
0924 07:14:33.956658 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:33.956659 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:33.956764 (+ 105us) server_negotiation.cc:407] Connection header received
0924 07:14:33.956818 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:33.956818 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:33.956828 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:33.956877 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:33.957195 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.957628 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.958255 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:33.958420 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:33.958460 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:33.959166 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:33.959167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:33.959167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:33.959170 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:33.960365 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:33.960628 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:33.960628 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:33.960629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:33.960672 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:33.960786 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:33.960786 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:33.960786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:33.960898 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:33.960942 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:33.961030 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 07:14:33.961083 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:14:33.962154 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 18459
I20260924 07:14:33.962257 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:33.962430 10763 external_mini_cluster.cc:1428] Setting key 4e8b8fefddfc6e3efaa88012ec61e48f
WARNING: no policy specified for oryx/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab.
Entry for principal oryx/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab.
I20260924 07:14:33.983469 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:35053
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab
--principal=oryx/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:38111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:34.059564 18599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:34.059836 18599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:34.059880 18599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:34.061043 18599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:34.061079 18599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:34.061100 18599 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 07:14:34.061128 18599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:34.062556 18599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab
--principal=oryx/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.130.254:35053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:34.062748 18599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:34.062928 18599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:34.067304 18607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:34.067425 18599 server_base.cc:1034] running on GCE node
W20260924 07:14:34.067308 18605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:34.067333 18604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:34.067740 18599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:34.067966 18599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:34.069123 18599 hybrid_clock.cc:648] HybridClock initialized: now 1790234074069110 us; error 27 us; skew 500 ppm
Sep 24 07:14:34 dist-test-slave-7q7m krb5kdc[18202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234074, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:34.071293 18599 init.cc:377] Logged in from keytab as oryx/127.10.130.195@KRBTEST.COM (short username oryx)
I20260924 07:14:34.071643 18599 webserver.cc:460] Webserver started at http://127.10.130.195:33639/ using document root <none> and password file <none>
I20260924 07:14:34.071851 18599 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:34.071897 18599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:34.072037 18599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:34.072957 18599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "0b37ca7d22c24e45a9f9abddf38779d2"
format_stamp: "Formatted at 2026-09-24 07:14:34 on dist-test-slave-7q7m"
server_key: "c18ab3771ef45a0443382544247eb9c9"
server_key_iv: "b2df5a94c0999e7f38859bf867227759"
server_key_version: "encryptionkey@0"
I20260924 07:14:34.073261 18599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "0b37ca7d22c24e45a9f9abddf38779d2"
format_stamp: "Formatted at 2026-09-24 07:14:34 on dist-test-slave-7q7m"
server_key: "c18ab3771ef45a0443382544247eb9c9"
server_key_iv: "b2df5a94c0999e7f38859bf867227759"
server_key_version: "encryptionkey@0"
I20260924 07:14:34.074375 18599 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:34.074977 18614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:34.075120 18599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:34.075186 18599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "0b37ca7d22c24e45a9f9abddf38779d2"
format_stamp: "Formatted at 2026-09-24 07:14:34 on dist-test-slave-7q7m"
server_key: "c18ab3771ef45a0443382544247eb9c9"
server_key_iv: "b2df5a94c0999e7f38859bf867227759"
server_key_version: "encryptionkey@0"
I20260924 07:14:34.075248 18599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:34.091351 18599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:34.092855 18599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:34.092970 18599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:34.093379 18599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:34.093420 18599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:34.093444 18599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:34.093461 18599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 07:14:34 dist-test-slave-7q7m krb5kdc[18202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234074, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
I20260924 07:14:34.099829 18599 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:39997
I20260924 07:14:34.099927 18729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:39997 every 8 connection(s)
I20260924 07:14:34.100209 18599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:34.101883 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.093753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:43533 (local address 127.10.130.254:35053)
0924 07:14:34.093811 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.093813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.093994 (+ 181us) server_negotiation.cc:407] Connection header received
0924 07:14:34.094568 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.094569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.094581 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.094642 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:34.095420 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.096151 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.097103 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.097303 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.097356 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.099540 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:34.099541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:34.099542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:34.099548 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:34.100932 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:34.101253 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:34.101254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:34.101254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:34.101302 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:34.101462 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:34.101463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:34.101463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:34.101548 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:34.101590 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.101766 (+ 176us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.101822 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 07:14:34.102003 18623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.093729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:35053 (local address 127.10.130.195:43533)
0924 07:14:34.093870 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.093912 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.094474 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.094712 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.094716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.095041 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:34.095316 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.095323 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.096230 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.096231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.097029 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.097031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.097388 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.097389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.097486 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.097902 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:34.097912 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:34.099460 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:34.100995 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:34.100999 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:34.101001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:34.101197 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:34.101348 (+ 151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:34.101348 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:34.101349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:34.101429 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:34.101586 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:34.101588 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:34.101682 (+ 94us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.101753 (+ 71us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.101855 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":66,"threads_started":1}
I20260924 07:14:34.102442 18730 heartbeater.cc:347] Connected to a master server at 127.10.130.254:35053
I20260924 07:14:34.102514 18730 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:34.102658 18730 heartbeater.cc:511] Master 127.10.130.254:35053 requested a full tablet report, sending...
I20260924 07:14:34.103013 18246 ts_manager.cc:194] Registered new tserver with Master: 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195:39997)
I20260924 07:14:34.103452 18246 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:43533
I20260924 07:14:34.107168 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 18599
I20260924 07:14:34.107280 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:34.107474 10763 external_mini_cluster.cc:1428] Setting key eba0995d34de702e69120f6e0e5493e3
I20260924 07:14:34.108296 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:14:34.108314 18736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.103281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:35053 (local address 127.10.130.195:51441)
0924 07:14:34.103423 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.103432 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.103477 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.103592 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.103592 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.103781 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:34.103834 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.103836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.104561 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.104561 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.105065 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.105066 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.105387 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.105388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.105432 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.105717 (+ 285us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:34.105720 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:34.105960 (+ 240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:34.107350 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:34.107351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:34.107352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:34.107514 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:34.107713 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:34.107714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:34.107714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:34.107780 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:34.108021 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:34.108021 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:34.108065 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.108134 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.108224 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260924 07:14:34.108317 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.103342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:51441 (local address 127.10.130.254:35053)
0924 07:14:34.103374 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.103376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.103482 (+ 106us) server_negotiation.cc:407] Connection header received
0924 07:14:34.103520 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.103521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.103529 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.103578 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:34.103911 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.104490 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.105141 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.105331 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.105381 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.106023 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:34.106023 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:34.106024 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:34.106029 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:34.107269 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:34.107588 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:34.107589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:34.107589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:34.107646 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:34.107829 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:34.107830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:34.107830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:34.107947 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:34.108029 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.108165 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.108233 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
Sep 24 07:14:34 dist-test-slave-7q7m krb5kdc[18202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234074, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 07:14:34 dist-test-slave-7q7m krb5kdc[18202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:34 dist-test-slave-7q7m krb5kdc[18202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
W20260924 07:14:34.127528 18315 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:34.122524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47910 (local address 127.10.130.254:35053)
0924 07:14:34.122574 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.122576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.122584 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:34.122639 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.122640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.122650 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.122692 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:34.123201 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.123691 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.124451 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.124632 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.124684 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.127455 (+ 2771us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47910: BlockingRecv error: recv got EOF from 127.0.0.1:47910 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Sep 24 07:14:34 dist-test-slave-7q7m krb5kdc[18202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234074, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
I20260924 07:14:34.133864 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.128270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47920 (local address 127.10.130.254:35053)
0924 07:14:34.128329 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.128330 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.128384 (+ 54us) server_negotiation.cc:407] Connection header received
0924 07:14:34.128422 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.128423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.128430 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.128471 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:34.128858 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.129295 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.129938 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.130101 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.130140 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.131927 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:34.131928 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:34.131928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:34.131935 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:34.133110 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:34.133360 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:34.133361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:34.133361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:34.133404 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:34.133521 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:34.133521 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:34.133521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:34.133605 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:34.133666 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.133738 (+ 72us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.133787 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 07:14:34.135092 18246 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:14:34.136653 18246 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:34.142741 18749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.138478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:46589 (local address 127.0.0.1:59668)
0924 07:14:34.138614 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.138660 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.138724 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.139106 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.139109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.139110 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:34.139171 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.139173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.140502 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.140502 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.141584 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.141587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.142472 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.142474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.142547 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.142551 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.142593 (+ 42us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.142652 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":1,"thread_start_us":73,"threads_started":1}
I20260924 07:14:34.142773 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.138777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57618 (local address 127.10.130.195:39997)
0924 07:14:34.138918 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.138923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.139045 (+ 122us) server_negotiation.cc:407] Connection header received
0924 07:14:34.139083 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.139085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.139099 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.139156 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:34.139347 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.140239 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.141664 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.142386 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.142474 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.142532 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.142625 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.142661 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260924 07:14:34.142746 18751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.138673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:39997 (local address 127.0.0.1:57618)
0924 07:14:34.138931 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.138940 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.138991 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.139210 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.139210 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.139212 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:34.139270 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.139272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.140313 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.140314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.141585 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.141587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.142458 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.142458 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.142535 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.142549 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.142615 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.142661 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":8,"thread_start_us":43,"threads_started":1}
I20260924 07:14:34.143213 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.138589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59668 (local address 127.10.130.193:46589)
0924 07:14:34.138918 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.138923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.138933 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:14:34.138974 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.138976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.138993 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.139096 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:34.139262 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.140436 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.141657 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.142392 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.142999 (+ 607us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.143061 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.143085 (+ 24us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.143123 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":121,"threads_started":1}
I20260924 07:14:34.143653 18664 tablet_service.cc:1467] Processing CreateTablet for tablet 2e306b21b44c4473ab316d72704e020c (DEFAULT_TABLE table=test-table [id=4cf5333c6c39467181be1f18d6069d7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:34.143786 18384 tablet_service.cc:1467] Processing CreateTablet for tablet 2e306b21b44c4473ab316d72704e020c (DEFAULT_TABLE table=test-table [id=4cf5333c6c39467181be1f18d6069d7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:34.144038 18664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e306b21b44c4473ab316d72704e020c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.144156 18384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e306b21b44c4473ab316d72704e020c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.144265 18752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.138792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:37809 (local address 127.0.0.1:45488)
0924 07:14:34.138966 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.138975 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.139018 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.139213 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.139213 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.139215 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:34.139270 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.139272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.141678 (+ 2406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.141679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.142339 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.142341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.144048 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.144049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.144110 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.144114 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.144165 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.144203 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":3,"thread_start_us":41,"threads_started":1}
I20260924 07:14:34.144510 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.138896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45488 (local address 127.10.130.194:37809)
0924 07:14:34.139082 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.139085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.139094 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:34.139125 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.139127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.139142 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.139220 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:34.139333 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.140423 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.142419 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.143384 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.143466 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.143516 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.144371 (+ 855us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.144414 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":90,"threads_started":1}
I20260924 07:14:34.145109 18524 tablet_service.cc:1467] Processing CreateTablet for tablet 2e306b21b44c4473ab316d72704e020c (DEFAULT_TABLE table=test-table [id=4cf5333c6c39467181be1f18d6069d7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:34.145426 18524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e306b21b44c4473ab316d72704e020c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.146880 18755 tablet_bootstrap.cc:492] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2: Bootstrap starting.
I20260924 07:14:34.147488 18755 tablet_bootstrap.cc:654] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.147712 18756 tablet_bootstrap.cc:492] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b: Bootstrap starting.
I20260924 07:14:34.147913 18755 log.cc:824] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:34.148277 18756 tablet_bootstrap.cc:654] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.148569 18756 log.cc:824] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:34.148680 18755 tablet_bootstrap.cc:492] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2: No bootstrap required, opened a new log
I20260924 07:14:34.148746 18755 ts_tablet_manager.cc:1397] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:14:34.148994 18759 tablet_bootstrap.cc:492] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70: Bootstrap starting.
I20260924 07:14:34.149245 18756 tablet_bootstrap.cc:492] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b: No bootstrap required, opened a new log
I20260924 07:14:34.149314 18756 ts_tablet_manager.cc:1397] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:34.149576 18759 tablet_bootstrap.cc:654] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.149873 18759 log.cc:824] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:34.150501 18755 raft_consensus.cc:348] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } }
I20260924 07:14:34.150614 18759 tablet_bootstrap.cc:492] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70: No bootstrap required, opened a new log
I20260924 07:14:34.150620 18755 raft_consensus.cc:374] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.150652 18755 raft_consensus.cc:729] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b37ca7d22c24e45a9f9abddf38779d2, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.150669 18759 ts_tablet_manager.cc:1397] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:34.150786 18755 consensus_queue.cc:260] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } }
I20260924 07:14:34.151003 18755 ts_tablet_manager.cc:1428] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 07:14:34.151113 18730 heartbeater.cc:503] Master 127.10.130.254:35053 was elected leader, sending a full tablet report...
I20260924 07:14:34.151175 18756 raft_consensus.cc:348] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } }
I20260924 07:14:34.151391 18756 raft_consensus.cc:374] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.151466 18756 raft_consensus.cc:729] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e1f2b0ba4aa42298262543d325ea10b, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.151635 18756 consensus_queue.cc:260] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } }
I20260924 07:14:34.151937 18756 ts_tablet_manager.cc:1428] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260924 07:14:34.151945 18450 heartbeater.cc:503] Master 127.10.130.254:35053 was elected leader, sending a full tablet report...
I20260924 07:14:34.152545 18759 raft_consensus.cc:348] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } }
I20260924 07:14:34.152686 18759 raft_consensus.cc:374] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.152729 18759 raft_consensus.cc:729] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ff4dd74c094d099b72951152543b70, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.152850 18759 consensus_queue.cc:260] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } }
I20260924 07:14:34.153062 18759 ts_tablet_manager.cc:1428] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 07:14:34.153100 18590 heartbeater.cc:503] Master 127.10.130.254:35053 was elected leader, sending a full tablet report...
W20260924 07:14:34.203855 18591 tablet.cc:2369] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 07:14:34.284096 18451 tablet.cc:2369] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:14:34.333521 18763 raft_consensus.cc:482] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:34.333669 18763 raft_consensus.cc:504] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } }
I20260924 07:14:34.333984 18763 leader_election.cc:288] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193:46589), 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195:39997)
I20260924 07:14:34.337818 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.334257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:37875 (local address 127.10.130.193:46589)
0924 07:14:34.334326 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.334328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.334339 (+ 11us) server_negotiation.cc:407] Connection header received
0924 07:14:34.334368 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.334369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.334380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.334455 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:34.334604 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.335368 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.336900 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.337466 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.337547 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.337554 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.337728 (+ 174us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.337765 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 07:14:34.337853 18483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.334164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:46589 (local address 127.10.130.194:37875)
0924 07:14:34.334216 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.334228 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.334271 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.334461 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.334462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.334464 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:34.334534 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.334536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.335455 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.335456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.336761 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.336764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.337550 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.337550 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.337655 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.337659 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.337709 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.337754 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":76}
I20260924 07:14:34.338217 18764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.334269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:39997 (local address 127.10.130.194:38057)
0924 07:14:34.334429 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.334437 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.334477 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.334598 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.334598 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.334600 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:34.334649 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.334650 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.335459 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.335460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.336761 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.336764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.337550 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.337550 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.338023 (+ 473us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.338027 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.338081 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.338152 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":38,"thread_start_us":93,"threads_started":1}
I20260924 07:14:34.338243 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.334395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:38057 (local address 127.10.130.195:39997)
0924 07:14:34.334457 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.334459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.334471 (+ 12us) server_negotiation.cc:407] Connection header received
0924 07:14:34.334525 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.334525 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.334533 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.334589 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:34.334709 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.335392 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.336913 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.337468 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.337547 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.337555 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.338128 (+ 573us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.338172 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 07:14:34.338191 18404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e306b21b44c4473ab316d72704e020c" candidate_uuid: "54ff4dd74c094d099b72951152543b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e1f2b0ba4aa42298262543d325ea10b" is_pre_election: true
I20260924 07:14:34.338343 18404 raft_consensus.cc:2454] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ff4dd74c094d099b72951152543b70 in term 0.
I20260924 07:14:34.338501 18684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e306b21b44c4473ab316d72704e020c" candidate_uuid: "54ff4dd74c094d099b72951152543b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" is_pre_election: true
I20260924 07:14:34.338642 18684 raft_consensus.cc:2454] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ff4dd74c094d099b72951152543b70 in term 0.
I20260924 07:14:34.338796 18477 leader_election.cc:302] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54ff4dd74c094d099b72951152543b70, 6e1f2b0ba4aa42298262543d325ea10b; no voters:
I20260924 07:14:34.338941 18763 raft_consensus.cc:2781] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:34.339002 18763 raft_consensus.cc:482] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:34.339018 18763 raft_consensus.cc:3037] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.339689 18763 raft_consensus.cc:504] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } }
I20260924 07:14:34.339798 18763 leader_election.cc:288] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [CANDIDATE]: Term 1 election: Requested vote from peers 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193:46589), 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195:39997)
I20260924 07:14:34.339941 18404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e306b21b44c4473ab316d72704e020c" candidate_uuid: "54ff4dd74c094d099b72951152543b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e1f2b0ba4aa42298262543d325ea10b"
I20260924 07:14:34.340008 18684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e306b21b44c4473ab316d72704e020c" candidate_uuid: "54ff4dd74c094d099b72951152543b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b37ca7d22c24e45a9f9abddf38779d2"
I20260924 07:14:34.340044 18404 raft_consensus.cc:3037] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.340065 18684 raft_consensus.cc:3037] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.340665 18404 raft_consensus.cc:2454] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ff4dd74c094d099b72951152543b70 in term 1.
I20260924 07:14:34.340683 18684 raft_consensus.cc:2454] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ff4dd74c094d099b72951152543b70 in term 1.
I20260924 07:14:34.340837 18477 leader_election.cc:302] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b37ca7d22c24e45a9f9abddf38779d2, 54ff4dd74c094d099b72951152543b70; no voters:
I20260924 07:14:34.340926 18763 raft_consensus.cc:2781] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:34.341065 18763 raft_consensus.cc:686] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [term 1 LEADER]: Becoming Leader. State: Replica: 54ff4dd74c094d099b72951152543b70, State: Running, Role: LEADER
I20260924 07:14:34.341140 18763 consensus_queue.cc:237] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } }
I20260924 07:14:34.341909 18247 catalog_manager.cc:5495] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54ff4dd74c094d099b72951152543b70 (127.10.130.194). New cstate: current_term: 1 leader_uuid: "54ff4dd74c094d099b72951152543b70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } health_report { overall_health: HEALTHY } } }
W20260924 07:14:34.351222 18731 tablet.cc:2369] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:14:34.365424 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.363236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.10.130.194:37809)
0924 07:14:34.363277 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.363279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.363287 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:34.363346 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.363347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.363365 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.363422 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:34.363527 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.364030 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.364758 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.364942 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.364994 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.365133 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:34.365229 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:34.365297 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.365334 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.365354 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:14:34.367027 18684 raft_consensus.cc:1261] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Refusing update from remote peer 54ff4dd74c094d099b72951152543b70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:34.367041 18404 raft_consensus.cc:1261] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Refusing update from remote peer 54ff4dd74c094d099b72951152543b70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:34.367328 18763 consensus_queue.cc:1035] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:34.367468 18765 consensus_queue.cc:1035] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:34.369637 18768 mvcc.cc:205] Tried to move back new op lower bound from 7332798768604905472 to 7332798768501702656. Current Snapshot: MvccSnapshot[applied={T|T < 7332798768604905472}]
I20260924 07:14:34.370389 18769 mvcc.cc:205] Tried to move back new op lower bound from 7332798768604905472 to 7332798768501702656. Current Snapshot: MvccSnapshot[applied={T|T < 7332798768604905472}]
I20260924 07:14:34.371416 18247 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:34.371492 18247 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:34.372155 18247 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c: Sending DeleteTablet for 3 replicas of tablet 2e306b21b44c4473ab316d72704e020c
I20260924 07:14:34.372443 18384 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e306b21b44c4473ab316d72704e020c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:34 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:59668
I20260924 07:14:34.372606 18664 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e306b21b44c4473ab316d72704e020c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:34 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:57618
I20260924 07:14:34.372583 18247 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 07:14:34.372677 18778 tablet_replica.cc:330] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b: stopping tablet replica
W20260924 07:14:34.372707 18247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:34.372763 18778 raft_consensus.cc:2229] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:34.372856 18778 raft_consensus.cc:2258] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:34.373117 18524 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e306b21b44c4473ab316d72704e020c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:34 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:45488
I20260924 07:14:34.373344 18780 tablet_replica.cc:330] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70: stopping tablet replica
I20260924 07:14:34.373592 18779 tablet_replica.cc:330] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2: stopping tablet replica
I20260924 07:14:34.373701 18779 raft_consensus.cc:2229] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:34.373831 18779 raft_consensus.cc:2258] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:34.373957 18384 tablet_service.cc:1467] Processing CreateTablet for tablet 80237620e8f64b2e81ece1fb83220b1b (DEFAULT_TABLE table=test-table [id=b40f1d45c42f48398330751e109545b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:34.374064 18384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80237620e8f64b2e81ece1fb83220b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.374099 18664 tablet_service.cc:1467] Processing CreateTablet for tablet 80237620e8f64b2e81ece1fb83220b1b (DEFAULT_TABLE table=test-table [id=b40f1d45c42f48398330751e109545b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:34.374202 18664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80237620e8f64b2e81ece1fb83220b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.374913 18756 tablet_bootstrap.cc:492] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b: Bootstrap starting.
I20260924 07:14:34.375190 18778 ts_tablet_manager.cc:1905] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:34.375278 18756 tablet_bootstrap.cc:654] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.375479 18755 tablet_bootstrap.cc:492] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2: Bootstrap starting.
I20260924 07:14:34.375873 18755 tablet_bootstrap.cc:654] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.375928 18779 ts_tablet_manager.cc:1905] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:34.376426 18755 tablet_bootstrap.cc:492] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2: No bootstrap required, opened a new log
I20260924 07:14:34.376475 18755 ts_tablet_manager.cc:1397] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:34.374039 18524 tablet_service.cc:1467] Processing CreateTablet for tablet 80237620e8f64b2e81ece1fb83220b1b (DEFAULT_TABLE table=test-table [id=b40f1d45c42f48398330751e109545b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:34.376610 18755 raft_consensus.cc:348] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.376667 18755 raft_consensus.cc:374] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.376685 18755 raft_consensus.cc:729] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b37ca7d22c24e45a9f9abddf38779d2, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.376698 18524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80237620e8f64b2e81ece1fb83220b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.376722 18755 consensus_queue.cc:260] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.376811 18755 ts_tablet_manager.cc:1428] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:34.377372 18779 ts_tablet_manager.cc:1918] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:34.377429 18779 log.cc:1196] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/2e306b21b44c4473ab316d72704e020c
I20260924 07:14:34.377710 18779 ts_tablet_manager.cc:1939] T 2e306b21b44c4473ab316d72704e020c P 0b37ca7d22c24e45a9f9abddf38779d2: Deleting consensus metadata
I20260924 07:14:34.377974 18759 tablet_bootstrap.cc:492] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70: Bootstrap starting.
I20260924 07:14:34.378201 18778 ts_tablet_manager.cc:1918] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:34.378293 18778 log.cc:1196] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/wal/wals/2e306b21b44c4473ab316d72704e020c
I20260924 07:14:34.378397 18759 tablet_bootstrap.cc:654] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.378536 18778 ts_tablet_manager.cc:1939] T 2e306b21b44c4473ab316d72704e020c P 6e1f2b0ba4aa42298262543d325ea10b: Deleting consensus metadata
I20260924 07:14:34.378829 18233 catalog_manager.cc:4841] TS 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195:39997): tablet 2e306b21b44c4473ab316d72704e020c (table test-table [id=4cf5333c6c39467181be1f18d6069d7b]) successfully deleted
I20260924 07:14:34.373478 18780 raft_consensus.cc:2229] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:34.378939 18233 catalog_manager.cc:4841] TS 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193:46589): tablet 2e306b21b44c4473ab316d72704e020c (table test-table [id=4cf5333c6c39467181be1f18d6069d7b]) successfully deleted
I20260924 07:14:34.378975 18780 raft_consensus.cc:2258] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:34.379179 18759 tablet_bootstrap.cc:492] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70: No bootstrap required, opened a new log
I20260924 07:14:34.379228 18759 ts_tablet_manager.cc:1397] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:34.379356 18759 raft_consensus.cc:348] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.379403 18780 ts_tablet_manager.cc:1905] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:34.379419 18759 raft_consensus.cc:374] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.379520 18759 raft_consensus.cc:729] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ff4dd74c094d099b72951152543b70, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.379563 18759 consensus_queue.cc:260] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.379649 18759 ts_tablet_manager.cc:1428] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:34.380602 18780 ts_tablet_manager.cc:1918] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:34.380661 18780 log.cc:1196] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/2e306b21b44c4473ab316d72704e020c
I20260924 07:14:34.380843 18780 ts_tablet_manager.cc:1939] T 2e306b21b44c4473ab316d72704e020c P 54ff4dd74c094d099b72951152543b70: Deleting consensus metadata
I20260924 07:14:34.381191 18756 tablet_bootstrap.cc:492] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b: No bootstrap required, opened a new log
I20260924 07:14:34.381233 18756 ts_tablet_manager.cc:1397] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260924 07:14:34.381228 18233 catalog_manager.cc:4841] TS 54ff4dd74c094d099b72951152543b70 (127.10.130.194:37809): tablet 2e306b21b44c4473ab316d72704e020c (table test-table [id=4cf5333c6c39467181be1f18d6069d7b]) successfully deleted
I20260924 07:14:34.381359 18756 raft_consensus.cc:348] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.381415 18756 raft_consensus.cc:374] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.381433 18756 raft_consensus.cc:729] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e1f2b0ba4aa42298262543d325ea10b, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.381469 18756 consensus_queue.cc:260] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.381560 18756 ts_tablet_manager.cc:1428] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:34.413128 18765 raft_consensus.cc:482] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:34.413193 18765 raft_consensus.cc:504] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.413329 18765 leader_election.cc:288] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193:46589), 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195:39997)
I20260924 07:14:34.413532 18684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80237620e8f64b2e81ece1fb83220b1b" candidate_uuid: "54ff4dd74c094d099b72951152543b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" is_pre_election: true
I20260924 07:14:34.413540 18404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80237620e8f64b2e81ece1fb83220b1b" candidate_uuid: "54ff4dd74c094d099b72951152543b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e1f2b0ba4aa42298262543d325ea10b" is_pre_election: true
I20260924 07:14:34.413605 18684 raft_consensus.cc:2454] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ff4dd74c094d099b72951152543b70 in term 0.
I20260924 07:14:34.413626 18404 raft_consensus.cc:2454] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ff4dd74c094d099b72951152543b70 in term 0.
I20260924 07:14:34.413741 18477 leader_election.cc:302] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b37ca7d22c24e45a9f9abddf38779d2, 54ff4dd74c094d099b72951152543b70; no voters:
I20260924 07:14:34.413820 18765 raft_consensus.cc:2781] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:34.413868 18765 raft_consensus.cc:482] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:34.413888 18765 raft_consensus.cc:3037] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.414525 18765 raft_consensus.cc:504] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.414616 18765 leader_election.cc:288] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [CANDIDATE]: Term 1 election: Requested vote from peers 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193:46589), 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195:39997)
I20260924 07:14:34.414777 18404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80237620e8f64b2e81ece1fb83220b1b" candidate_uuid: "54ff4dd74c094d099b72951152543b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e1f2b0ba4aa42298262543d325ea10b"
I20260924 07:14:34.414841 18404 raft_consensus.cc:3037] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.414794 18684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80237620e8f64b2e81ece1fb83220b1b" candidate_uuid: "54ff4dd74c094d099b72951152543b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b37ca7d22c24e45a9f9abddf38779d2"
I20260924 07:14:34.414883 18684 raft_consensus.cc:3037] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.415422 18684 raft_consensus.cc:2454] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ff4dd74c094d099b72951152543b70 in term 1.
I20260924 07:14:34.415575 18477 leader_election.cc:302] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b37ca7d22c24e45a9f9abddf38779d2, 54ff4dd74c094d099b72951152543b70; no voters:
I20260924 07:14:34.415653 18765 raft_consensus.cc:2781] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:34.415694 18765 raft_consensus.cc:686] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [term 1 LEADER]: Becoming Leader. State: Replica: 54ff4dd74c094d099b72951152543b70, State: Running, Role: LEADER
I20260924 07:14:34.415728 18765 consensus_queue.cc:237] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.415853 18404 raft_consensus.cc:2454] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ff4dd74c094d099b72951152543b70 in term 1.
I20260924 07:14:34.416255 18246 catalog_manager.cc:5495] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54ff4dd74c094d099b72951152543b70 (127.10.130.194). New cstate: current_term: 1 leader_uuid: "54ff4dd74c094d099b72951152543b70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } health_report { overall_health: UNKNOWN } } }
Sep 24 07:14:34 dist-test-slave-7q7m krb5kdc[18202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234073, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.254@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
I20260924 07:14:34.451005 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.444643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47932 (local address 127.10.130.254:35053)
0924 07:14:34.444694 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.444696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.444812 (+ 116us) server_negotiation.cc:407] Connection header received
0924 07:14:34.444867 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.444868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.444878 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.444924 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:34.445328 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.445800 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.446447 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.446618 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.446668 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.448717 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:34.448719 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:34.448719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:34.448725 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:34.449929 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:34.450341 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:34.450342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:34.450342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:34.450387 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:34.450566 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:34.450567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:34.450567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:34.450657 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:34.450699 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.450883 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.450945 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 07:14:34.451112 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.444600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:35053 (local address 127.0.0.1:47932)
0924 07:14:34.444747 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.444755 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.444816 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.444937 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.444938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.445213 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:34.445253 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.445255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.445863 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.445864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.446377 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.446379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.446670 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.446671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.446717 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.447238 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:34.447242 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:34.448636 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:34.450009 (+ 1373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:34.450016 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:34.450018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:34.450275 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:34.450440 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:34.450441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:34.450441 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:34.450513 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:34.450711 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:34.450717 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:34.450780 (+ 63us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.450864 (+ 84us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.451013 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":39,"thread_start_us":85,"threads_started":1}
I20260924 07:14:34.452940 18246 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:47932:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 07:14:34.453140 18246 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:34.454526 18524 tablet_service.cc:1467] Processing CreateTablet for tablet 8dbcf57762d5410e8ce82b0b31ad9f27 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c95baaff8ecf42c3ac79d895465304a3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 07:14:34.454535 18384 tablet_service.cc:1467] Processing CreateTablet for tablet 8dbcf57762d5410e8ce82b0b31ad9f27 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c95baaff8ecf42c3ac79d895465304a3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 07:14:34.454566 18664 tablet_service.cc:1467] Processing CreateTablet for tablet 8dbcf57762d5410e8ce82b0b31ad9f27 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c95baaff8ecf42c3ac79d895465304a3]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 07:14:34.454664 18384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dbcf57762d5410e8ce82b0b31ad9f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.454663 18524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dbcf57762d5410e8ce82b0b31ad9f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.454700 18664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8dbcf57762d5410e8ce82b0b31ad9f27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.456136 18756 tablet_bootstrap.cc:492] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b: Bootstrap starting.
I20260924 07:14:34.456210 18755 tablet_bootstrap.cc:492] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2: Bootstrap starting.
I20260924 07:14:34.456491 18756 tablet_bootstrap.cc:654] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.456593 18755 tablet_bootstrap.cc:654] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.456805 18759 tablet_bootstrap.cc:492] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70: Bootstrap starting.
I20260924 07:14:34.457160 18759 tablet_bootstrap.cc:654] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.457285 18755 tablet_bootstrap.cc:492] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2: No bootstrap required, opened a new log
I20260924 07:14:34.457334 18755 ts_tablet_manager.cc:1397] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:34.457476 18755 raft_consensus.cc:348] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.457543 18755 raft_consensus.cc:374] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.457562 18755 raft_consensus.cc:729] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b37ca7d22c24e45a9f9abddf38779d2, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.457605 18755 consensus_queue.cc:260] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.457690 18755 ts_tablet_manager.cc:1428] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:34.457811 18761 tablet_replica.cc:439] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } }
I20260924 07:14:34.457875 18756 tablet_bootstrap.cc:492] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b: No bootstrap required, opened a new log
I20260924 07:14:34.457885 18761 tablet_replica.cc:442] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:34.457916 18756 ts_tablet_manager.cc:1397] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:14:34.458041 18756 raft_consensus.cc:348] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.458099 18756 raft_consensus.cc:374] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.458117 18756 raft_consensus.cc:729] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e1f2b0ba4aa42298262543d325ea10b, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.458149 18756 consensus_queue.cc:260] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.458236 18756 ts_tablet_manager.cc:1428] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:34.458367 18762 tablet_replica.cc:439] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } }
I20260924 07:14:34.458437 18762 tablet_replica.cc:442] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:34.458879 18759 tablet_bootstrap.cc:492] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70: No bootstrap required, opened a new log
I20260924 07:14:34.458925 18759 ts_tablet_manager.cc:1397] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:34.459054 18759 raft_consensus.cc:348] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.459112 18759 raft_consensus.cc:374] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.459129 18759 raft_consensus.cc:729] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ff4dd74c094d099b72951152543b70, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.459177 18759 consensus_queue.cc:260] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.459261 18759 ts_tablet_manager.cc:1428] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:34.459331 18765 tablet_replica.cc:439] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } }
I20260924 07:14:34.459412 18765 tablet_replica.cc:442] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:34.722112 18762 raft_consensus.cc:482] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:34.722216 18762 raft_consensus.cc:504] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.722524 18762 leader_election.cc:288] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54ff4dd74c094d099b72951152543b70 (127.10.130.194:37809), 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195:39997)
I20260924 07:14:34.726404 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.722995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:51025 (local address 127.10.130.195:39997)
0924 07:14:34.723036 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.723037 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.723065 (+ 28us) server_negotiation.cc:407] Connection header received
0924 07:14:34.723119 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.723120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.723127 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.723174 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:34.723296 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.724032 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.725510 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.726056 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.726129 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.726136 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.726301 (+ 165us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.726336 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 07:14:34.726404 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.722761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:59911 (local address 127.10.130.194:37809)
0924 07:14:34.722811 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.722813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.722934 (+ 121us) server_negotiation.cc:407] Connection header received
0924 07:14:34.723006 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.723007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.723019 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.723098 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:34.723245 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.724033 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.725509 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.726073 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.726128 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.726136 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.726301 (+ 165us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.726339 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:14:34.726465 18794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.722720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:37809 (local address 127.10.130.193:59911)
0924 07:14:34.722867 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.722878 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.722943 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.723109 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.723110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.723112 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:34.723184 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.723186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.724100 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.724102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.725409 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.725413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.726138 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.726138 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.726238 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.726242 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.726290 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.726342 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":63,"thread_start_us":72,"threads_started":1}
I20260924 07:14:34.726469 18795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.722904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:39997 (local address 127.10.130.193:51025)
0924 07:14:34.723022 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.723029 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.723072 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.723183 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.723184 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.723185 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:34.723248 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.723249 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.724104 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.724104 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.725410 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.725413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.726136 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.726137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.726238 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.726242 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.726290 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.726343 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":60,"thread_start_us":56,"threads_started":1}
I20260924 07:14:34.726684 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dbcf57762d5410e8ce82b0b31ad9f27" candidate_uuid: "6e1f2b0ba4aa42298262543d325ea10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ff4dd74c094d099b72951152543b70" is_pre_election: true
I20260924 07:14:34.726819 18544 raft_consensus.cc:2454] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e1f2b0ba4aa42298262543d325ea10b in term 0.
I20260924 07:14:34.726886 18684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dbcf57762d5410e8ce82b0b31ad9f27" candidate_uuid: "6e1f2b0ba4aa42298262543d325ea10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" is_pre_election: true
I20260924 07:14:34.726970 18684 raft_consensus.cc:2454] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e1f2b0ba4aa42298262543d325ea10b in term 0.
I20260924 07:14:34.727025 18337 leader_election.cc:302] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54ff4dd74c094d099b72951152543b70, 6e1f2b0ba4aa42298262543d325ea10b; no voters:
I20260924 07:14:34.727175 18762 raft_consensus.cc:2781] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:34.727236 18762 raft_consensus.cc:482] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:34.727255 18762 raft_consensus.cc:3037] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.727921 18762 raft_consensus.cc:504] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.728052 18762 leader_election.cc:288] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [CANDIDATE]: Term 1 election: Requested vote from peers 54ff4dd74c094d099b72951152543b70 (127.10.130.194:37809), 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195:39997)
I20260924 07:14:34.728206 18684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dbcf57762d5410e8ce82b0b31ad9f27" candidate_uuid: "6e1f2b0ba4aa42298262543d325ea10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b37ca7d22c24e45a9f9abddf38779d2"
I20260924 07:14:34.728273 18684 raft_consensus.cc:3037] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.728755 18684 raft_consensus.cc:2454] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e1f2b0ba4aa42298262543d325ea10b in term 1.
I20260924 07:14:34.728859 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8dbcf57762d5410e8ce82b0b31ad9f27" candidate_uuid: "6e1f2b0ba4aa42298262543d325ea10b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ff4dd74c094d099b72951152543b70"
I20260924 07:14:34.728904 18337 leader_election.cc:302] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b37ca7d22c24e45a9f9abddf38779d2, 6e1f2b0ba4aa42298262543d325ea10b; no voters:
I20260924 07:14:34.728935 18544 raft_consensus.cc:3037] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.728986 18762 raft_consensus.cc:2781] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:34.729133 18762 raft_consensus.cc:686] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [term 1 LEADER]: Becoming Leader. State: Replica: 6e1f2b0ba4aa42298262543d325ea10b, State: Running, Role: LEADER
I20260924 07:14:34.729195 18762 consensus_queue.cc:237] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.729701 18544 raft_consensus.cc:2454] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e1f2b0ba4aa42298262543d325ea10b in term 1.
I20260924 07:14:34.729697 18796 tablet_replica.cc:439] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b: TxnStatusTablet state changed. Reason: New leader 6e1f2b0ba4aa42298262543d325ea10b. Latest consensus state: current_term: 1 leader_uuid: "6e1f2b0ba4aa42298262543d325ea10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } }
I20260924 07:14:34.729780 18796 tablet_replica.cc:442] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b: This TxnStatusTablet replica's current role is: LEADER
I20260924 07:14:34.729866 18246 catalog_manager.cc:5495] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193). New cstate: current_term: 1 leader_uuid: "6e1f2b0ba4aa42298262543d325ea10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } health_report { overall_health: UNKNOWN } } }
I20260924 07:14:34.730170 18798 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 07:14:34.730262 18798 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 07:14:34.741480 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.738835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:46589 (local address 127.0.0.1:59678)
0924 07:14:34.738962 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.738971 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.739015 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.739149 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.739150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.739151 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 07:14:34.739187 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.739189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.740066 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.740066 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.740613 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.740614 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.741032 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.741033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.741076 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.741088 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 07:14:34.741360 (+ 272us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 07:14:34.741364 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.741395 (+ 31us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.741427 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260924 07:14:34.741485 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.738905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59678 (local address 127.10.130.193:46589)
0924 07:14:34.738947 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.738949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.739002 (+ 53us) server_negotiation.cc:407] Connection header received
0924 07:14:34.739065 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.739066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.739081 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.739145 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:34.739243 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.739947 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.740676 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.740972 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.741033 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.741158 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:34.741297 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:34.741368 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.741401 (+ 33us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.741419 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:14:34.742338 18684 raft_consensus.cc:1261] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Refusing update from remote peer 6e1f2b0ba4aa42298262543d325ea10b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:34.742451 18544 raft_consensus.cc:1261] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Refusing update from remote peer 6e1f2b0ba4aa42298262543d325ea10b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:34.742558 18796 consensus_queue.cc:1035] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:34.742654 18762 consensus_queue.cc:1035] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b [LEADER]: Connected to new peer: Peer: permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:34.743160 18761 tablet_replica.cc:439] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2: TxnStatusTablet state changed. Reason: New leader 6e1f2b0ba4aa42298262543d325ea10b. Latest consensus state: current_term: 1 leader_uuid: "6e1f2b0ba4aa42298262543d325ea10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } }
I20260924 07:14:34.743209 18765 tablet_replica.cc:439] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70: TxnStatusTablet state changed. Reason: New leader 6e1f2b0ba4aa42298262543d325ea10b. Latest consensus state: current_term: 1 leader_uuid: "6e1f2b0ba4aa42298262543d325ea10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } }
I20260924 07:14:34.743253 18761 tablet_replica.cc:442] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:34.743271 18765 tablet_replica.cc:442] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:34.743948 18762 tablet_replica.cc:439] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e1f2b0ba4aa42298262543d325ea10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } }
I20260924 07:14:34.744035 18762 tablet_replica.cc:442] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b: This TxnStatusTablet replica's current role is: LEADER
I20260924 07:14:34.744205 18761 tablet_replica.cc:439] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e1f2b0ba4aa42298262543d325ea10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } }
I20260924 07:14:34.744287 18761 tablet_replica.cc:442] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 0b37ca7d22c24e45a9f9abddf38779d2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:34.744933 18762 tablet_replica.cc:439] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e1f2b0ba4aa42298262543d325ea10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } }
I20260924 07:14:34.745008 18762 tablet_replica.cc:442] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 6e1f2b0ba4aa42298262543d325ea10b: This TxnStatusTablet replica's current role is: LEADER
I20260924 07:14:34.745426 18765 tablet_replica.cc:439] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e1f2b0ba4aa42298262543d325ea10b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } } }
I20260924 07:14:34.745503 18765 tablet_replica.cc:442] T 8dbcf57762d5410e8ce82b0b31ad9f27 P 54ff4dd74c094d099b72951152543b70: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 07:14:34.749361 18805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.746977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:46589 (local address 127.10.130.194:49221)
0924 07:14:34.747067 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.747076 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.747127 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.747240 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.747240 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.747241 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 07:14:34.747275 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.747276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.748094 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.748094 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.748639 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.748641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.749065 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.749065 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.749100 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.749110 (+ 10us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 07:14:34.749248 (+ 138us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 07:14:34.749249 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.749275 (+ 26us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.749308 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"thread_start_us":45,"threads_started":1}
I20260924 07:14:34.749358 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.747037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:49221 (local address 127.10.130.193:46589)
0924 07:14:34.747069 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.747071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.747114 (+ 43us) server_negotiation.cc:407] Connection header received
0924 07:14:34.747163 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.747164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.747172 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.747238 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:34.747308 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.748030 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.748701 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.749003 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.749057 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.749166 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:34.749201 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:34.749240 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.749280 (+ 40us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.749296 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:14:34.751271 18684 raft_consensus.cc:1261] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Refusing update from remote peer 54ff4dd74c094d099b72951152543b70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:34.751283 18404 raft_consensus.cc:1261] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Refusing update from remote peer 54ff4dd74c094d099b72951152543b70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:34.751467 18765 consensus_queue.cc:1035] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:34.751555 18763 consensus_queue.cc:1035] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:34.752974 18770 mvcc.cc:205] Tried to move back new op lower bound from 7332798770179870720 to 7332798768807211008. Current Snapshot: MvccSnapshot[applied={T|T < 7332798770179870720}]
I20260924 07:14:34.759253 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.756631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:39133 (local address 127.10.130.194:37809)
0924 07:14:34.756665 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.756667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.756801 (+ 134us) server_negotiation.cc:407] Connection header received
0924 07:14:34.756843 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.756844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.756852 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.756914 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:34.756997 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.757720 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.758510 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.758810 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.758897 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.759044 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:34.759077 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:34.759123 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.759158 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.759177 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:14:34.759277 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.756572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:37809 (local address 127.10.130.193:39133)
0924 07:14:34.756735 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.756745 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.756800 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.756916 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.756916 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.756917 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 07:14:34.756963 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.756964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.757791 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.757792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.758441 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.758443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.758912 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.758913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.758968 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.758981 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 07:14:34.759125 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 07:14:34.759126 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.759153 (+ 27us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.759200 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":61,"threads_started":1}
I20260924 07:14:34.766786 18247 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:34.766880 18247 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:34.767397 18247 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c: Sending DeleteTablet for 3 replicas of tablet 80237620e8f64b2e81ece1fb83220b1b
I20260924 07:14:34.767616 18384 tablet_service.cc:1514] Processing DeleteTablet for tablet 80237620e8f64b2e81ece1fb83220b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:34 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:59668
I20260924 07:14:34.767669 18664 tablet_service.cc:1514] Processing DeleteTablet for tablet 80237620e8f64b2e81ece1fb83220b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:34 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:57618
I20260924 07:14:34.767692 18778 tablet_replica.cc:330] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b: stopping tablet replica
I20260924 07:14:34.767727 18778 raft_consensus.cc:2229] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:34.767738 18779 tablet_replica.cc:330] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2: stopping tablet replica
I20260924 07:14:34.767755 18778 raft_consensus.cc:2258] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:34.767735 18524 tablet_service.cc:1514] Processing DeleteTablet for tablet 80237620e8f64b2e81ece1fb83220b1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:34 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:45488
I20260924 07:14:34.767778 18779 raft_consensus.cc:2229] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:34.767807 18779 raft_consensus.cc:2258] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:34.767808 18780 tablet_replica.cc:330] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70: stopping tablet replica
I20260924 07:14:34.767849 18780 raft_consensus.cc:2229] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:34.767894 18780 raft_consensus.cc:2258] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:34.768169 18780 ts_tablet_manager.cc:1905] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:34.768178 18779 ts_tablet_manager.cc:1905] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:34.770388 18778 ts_tablet_manager.cc:1905] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:34.770598 18779 ts_tablet_manager.cc:1918] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:34.770649 18779 log.cc:1196] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/80237620e8f64b2e81ece1fb83220b1b
I20260924 07:14:34.770830 18779 ts_tablet_manager.cc:1939] T 80237620e8f64b2e81ece1fb83220b1b P 0b37ca7d22c24e45a9f9abddf38779d2: Deleting consensus metadata
I20260924 07:14:34.771067 18780 ts_tablet_manager.cc:1918] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:34.771095 18233 catalog_manager.cc:4841] TS 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195:39997): tablet 80237620e8f64b2e81ece1fb83220b1b (table test-table [id=b40f1d45c42f48398330751e109545b3]) successfully deleted
I20260924 07:14:34.771128 18780 log.cc:1196] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/80237620e8f64b2e81ece1fb83220b1b
I20260924 07:14:34.771286 18780 ts_tablet_manager.cc:1939] T 80237620e8f64b2e81ece1fb83220b1b P 54ff4dd74c094d099b72951152543b70: Deleting consensus metadata
I20260924 07:14:34.771541 18233 catalog_manager.cc:4841] TS 54ff4dd74c094d099b72951152543b70 (127.10.130.194:37809): tablet 80237620e8f64b2e81ece1fb83220b1b (table test-table [id=b40f1d45c42f48398330751e109545b3]) successfully deleted
I20260924 07:14:34.771701 18778 ts_tablet_manager.cc:1918] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:34.771746 18778 log.cc:1196] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/wal/wals/80237620e8f64b2e81ece1fb83220b1b
I20260924 07:14:34.771911 18778 ts_tablet_manager.cc:1939] T 80237620e8f64b2e81ece1fb83220b1b P 6e1f2b0ba4aa42298262543d325ea10b: Deleting consensus metadata
I20260924 07:14:34.772244 18233 catalog_manager.cc:4841] TS 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193:46589): tablet 80237620e8f64b2e81ece1fb83220b1b (table test-table [id=b40f1d45c42f48398330751e109545b3]) successfully deleted
Sep 24 07:14:34 dist-test-slave-7q7m krb5kdc[18202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 07:14:34 dist-test-slave-7q7m krb5kdc[18202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:34 dist-test-slave-7q7m krb5kdc[18202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
W20260924 07:14:34.791193 18315 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:34.785855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47940 (local address 127.10.130.254:35053)
0924 07:14:34.785896 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.785898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.786059 (+ 161us) server_negotiation.cc:407] Connection header received
0924 07:14:34.786102 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.786103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.786113 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.786168 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:34.786670 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.787512 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.788276 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.788587 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.788655 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.791138 (+ 2483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47940: BlockingRecv error: recv got EOF from 127.0.0.1:47940 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 24 07:14:34 dist-test-slave-7q7m krb5kdc[18202](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM
I20260924 07:14:34.798597 18315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.792650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47952 (local address 127.10.130.254:35053)
0924 07:14:34.792702 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.792704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.792710 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:14:34.792725 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.792726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.792732 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.792774 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:34.793181 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.793628 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.794248 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.794440 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.794490 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.796548 (+ 2058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:34.796549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:34.796549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:34.796556 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:34.797679 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:34.797974 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:34.797975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:34.797976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:34.798023 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:34.798179 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:34.798179 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:34.798180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:34.798273 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:34.798362 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.798449 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.798518 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 07:14:34.799779 18247 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:14:34.799924 18247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:34.801132 18384 tablet_service.cc:1467] Processing CreateTablet for tablet 65453353fe924306a4154ccddcab5a67 (DEFAULT_TABLE table=test-table [id=c794f7e39e5b420f84f1d08ed5c94947]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:34.801189 18524 tablet_service.cc:1467] Processing CreateTablet for tablet 65453353fe924306a4154ccddcab5a67 (DEFAULT_TABLE table=test-table [id=c794f7e39e5b420f84f1d08ed5c94947]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:34.801251 18384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65453353fe924306a4154ccddcab5a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.801285 18524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65453353fe924306a4154ccddcab5a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.801283 18664 tablet_service.cc:1467] Processing CreateTablet for tablet 65453353fe924306a4154ccddcab5a67 (DEFAULT_TABLE table=test-table [id=c794f7e39e5b420f84f1d08ed5c94947]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:34.801398 18664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65453353fe924306a4154ccddcab5a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.802498 18759 tablet_bootstrap.cc:492] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70: Bootstrap starting.
I20260924 07:14:34.802510 18756 tablet_bootstrap.cc:492] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b: Bootstrap starting.
I20260924 07:14:34.802615 18755 tablet_bootstrap.cc:492] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2: Bootstrap starting.
I20260924 07:14:34.802778 18756 tablet_bootstrap.cc:654] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.802778 18759 tablet_bootstrap.cc:654] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.802985 18755 tablet_bootstrap.cc:654] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.803320 18756 tablet_bootstrap.cc:492] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b: No bootstrap required, opened a new log
I20260924 07:14:34.803367 18756 ts_tablet_manager.cc:1397] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 07:14:34.803501 18756 raft_consensus.cc:348] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.803579 18756 raft_consensus.cc:374] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.803583 18755 tablet_bootstrap.cc:492] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2: No bootstrap required, opened a new log
I20260924 07:14:34.803599 18756 raft_consensus.cc:729] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e1f2b0ba4aa42298262543d325ea10b, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.803622 18755 ts_tablet_manager.cc:1397] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:34.803637 18756 consensus_queue.cc:260] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.803710 18756 ts_tablet_manager.cc:1428] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:34.803753 18755 raft_consensus.cc:348] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.803824 18755 raft_consensus.cc:374] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.803851 18755 raft_consensus.cc:729] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b37ca7d22c24e45a9f9abddf38779d2, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.803894 18755 consensus_queue.cc:260] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.804013 18755 ts_tablet_manager.cc:1428] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:34.805083 18759 tablet_bootstrap.cc:492] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70: No bootstrap required, opened a new log
I20260924 07:14:34.805127 18759 ts_tablet_manager.cc:1397] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 07:14:34.805222 18759 raft_consensus.cc:348] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.805272 18759 raft_consensus.cc:374] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.805289 18759 raft_consensus.cc:729] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ff4dd74c094d099b72951152543b70, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.805326 18759 consensus_queue.cc:260] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.805408 18759 ts_tablet_manager.cc:1428] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:34.910670 18765 raft_consensus.cc:482] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:34.910763 18765 raft_consensus.cc:504] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.910914 18765 leader_election.cc:288] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193:46589), 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195:39997)
I20260924 07:14:34.911126 18404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65453353fe924306a4154ccddcab5a67" candidate_uuid: "54ff4dd74c094d099b72951152543b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e1f2b0ba4aa42298262543d325ea10b" is_pre_election: true
I20260924 07:14:34.911267 18404 raft_consensus.cc:2454] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ff4dd74c094d099b72951152543b70 in term 0.
I20260924 07:14:34.911268 18683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65453353fe924306a4154ccddcab5a67" candidate_uuid: "54ff4dd74c094d099b72951152543b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" is_pre_election: true
I20260924 07:14:34.911351 18683 raft_consensus.cc:2454] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54ff4dd74c094d099b72951152543b70 in term 0.
I20260924 07:14:34.911424 18477 leader_election.cc:302] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54ff4dd74c094d099b72951152543b70, 6e1f2b0ba4aa42298262543d325ea10b; no voters:
I20260924 07:14:34.911515 18765 raft_consensus.cc:2781] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:34.911556 18765 raft_consensus.cc:482] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:34.911574 18765 raft_consensus.cc:3037] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.912272 18765 raft_consensus.cc:504] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.912379 18765 leader_election.cc:288] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [CANDIDATE]: Term 1 election: Requested vote from peers 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193:46589), 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195:39997)
I20260924 07:14:34.912544 18404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65453353fe924306a4154ccddcab5a67" candidate_uuid: "54ff4dd74c094d099b72951152543b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e1f2b0ba4aa42298262543d325ea10b"
I20260924 07:14:34.912559 18683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65453353fe924306a4154ccddcab5a67" candidate_uuid: "54ff4dd74c094d099b72951152543b70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b37ca7d22c24e45a9f9abddf38779d2"
I20260924 07:14:34.912626 18683 raft_consensus.cc:3037] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.912626 18404 raft_consensus.cc:3037] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.913332 18683 raft_consensus.cc:2454] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ff4dd74c094d099b72951152543b70 in term 1.
I20260924 07:14:34.913332 18404 raft_consensus.cc:2454] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54ff4dd74c094d099b72951152543b70 in term 1.
I20260924 07:14:34.913484 18477 leader_election.cc:302] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54ff4dd74c094d099b72951152543b70, 6e1f2b0ba4aa42298262543d325ea10b; no voters:
I20260924 07:14:34.913574 18765 raft_consensus.cc:2781] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:34.913625 18765 raft_consensus.cc:686] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [term 1 LEADER]: Becoming Leader. State: Replica: 54ff4dd74c094d099b72951152543b70, State: Running, Role: LEADER
I20260924 07:14:34.913671 18765 consensus_queue.cc:237] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.914217 18246 catalog_manager.cc:5495] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54ff4dd74c094d099b72951152543b70 (127.10.130.194). New cstate: current_term: 1 leader_uuid: "54ff4dd74c094d099b72951152543b70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } health_report { overall_health: UNKNOWN } } }
I20260924 07:14:34.921811 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.919788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45518 (local address 127.10.130.194:37809)
0924 07:14:34.919832 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.919834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.919843 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:34.919896 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.919897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.919909 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.919951 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:34.920084 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.920538 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.921249 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.921431 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.921482 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.921606 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:34.921632 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:34.921670 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.921725 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.921747 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:14:34.922360 18683 raft_consensus.cc:1261] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Refusing update from remote peer 54ff4dd74c094d099b72951152543b70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:34.922397 18404 raft_consensus.cc:1261] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Refusing update from remote peer 54ff4dd74c094d099b72951152543b70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:34.922583 18773 consensus_queue.cc:1035] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:34.922698 18765 consensus_queue.cc:1035] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:34.924646 18247 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:34.924734 18247 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:34.925220 18247 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c: Sending DeleteTablet for 3 replicas of tablet 65453353fe924306a4154ccddcab5a67
I20260924 07:14:34.925420 18384 tablet_service.cc:1514] Processing DeleteTablet for tablet 65453353fe924306a4154ccddcab5a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:34 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:59668
I20260924 07:14:34.925428 18524 tablet_service.cc:1514] Processing DeleteTablet for tablet 65453353fe924306a4154ccddcab5a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:34 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:45488
I20260924 07:14:34.925521 18778 tablet_replica.cc:330] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b: stopping tablet replica
I20260924 07:14:34.925514 18664 tablet_service.cc:1514] Processing DeleteTablet for tablet 65453353fe924306a4154ccddcab5a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:34 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:57618
I20260924 07:14:34.925585 18778 raft_consensus.cc:2229] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:34.925592 18780 tablet_replica.cc:330] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70: stopping tablet replica
I20260924 07:14:34.925629 18778 raft_consensus.cc:2258] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:34.925635 18780 raft_consensus.cc:2229] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:34.925580 18247 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 07:14:34.925685 18780 raft_consensus.cc:2258] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 07:14:34.925694 18247 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:34.925746 18779 tablet_replica.cc:330] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2: stopping tablet replica
I20260924 07:14:34.925797 18779 raft_consensus.cc:2229] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:34.925834 18779 raft_consensus.cc:2258] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:34.926105 18780 ts_tablet_manager.cc:1905] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:34.926110 18778 ts_tablet_manager.cc:1905] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:34.927392 18779 ts_tablet_manager.cc:1905] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:34.927656 18384 tablet_service.cc:1467] Processing CreateTablet for tablet 6ddd9ac4d21442c3af246bff1bceaff7 (DEFAULT_TABLE table=test-table [id=937085ca085d4f5e92b41de9f849e264]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:34.927706 18524 tablet_service.cc:1467] Processing CreateTablet for tablet 6ddd9ac4d21442c3af246bff1bceaff7 (DEFAULT_TABLE table=test-table [id=937085ca085d4f5e92b41de9f849e264]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:34.927731 18664 tablet_service.cc:1467] Processing CreateTablet for tablet 6ddd9ac4d21442c3af246bff1bceaff7 (DEFAULT_TABLE table=test-table [id=937085ca085d4f5e92b41de9f849e264]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:34.927789 18778 ts_tablet_manager.cc:1918] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:34.927826 18778 log.cc:1196] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-0/wal/wals/65453353fe924306a4154ccddcab5a67
I20260924 07:14:34.927835 18664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ddd9ac4d21442c3af246bff1bceaff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.928006 18778 ts_tablet_manager.cc:1939] T 65453353fe924306a4154ccddcab5a67 P 6e1f2b0ba4aa42298262543d325ea10b: Deleting consensus metadata
I20260924 07:14:34.928120 18384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ddd9ac4d21442c3af246bff1bceaff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.928651 18779 ts_tablet_manager.cc:1918] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:34.928692 18779 log.cc:1196] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/65453353fe924306a4154ccddcab5a67
I20260924 07:14:34.928874 18779 ts_tablet_manager.cc:1939] T 65453353fe924306a4154ccddcab5a67 P 0b37ca7d22c24e45a9f9abddf38779d2: Deleting consensus metadata
I20260924 07:14:34.928887 18755 tablet_bootstrap.cc:492] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2: Bootstrap starting.
I20260924 07:14:34.928937 18756 tablet_bootstrap.cc:492] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b: Bootstrap starting.
I20260924 07:14:34.928972 18233 catalog_manager.cc:4841] TS 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193:46589): tablet 65453353fe924306a4154ccddcab5a67 (table test-table [id=c794f7e39e5b420f84f1d08ed5c94947]) successfully deleted
I20260924 07:14:34.929014 18780 ts_tablet_manager.cc:1918] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:34.929059 18780 log.cc:1196] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/65453353fe924306a4154ccddcab5a67
I20260924 07:14:34.929206 18756 tablet_bootstrap.cc:654] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.929247 18780 ts_tablet_manager.cc:1939] T 65453353fe924306a4154ccddcab5a67 P 54ff4dd74c094d099b72951152543b70: Deleting consensus metadata
I20260924 07:14:34.929245 18755 tablet_bootstrap.cc:654] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.929369 18524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ddd9ac4d21442c3af246bff1bceaff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:34.929630 18756 tablet_bootstrap.cc:492] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b: No bootstrap required, opened a new log
I20260924 07:14:34.929663 18756 ts_tablet_manager.cc:1397] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 07:14:34.929770 18756 raft_consensus.cc:348] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.929831 18756 raft_consensus.cc:374] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.929847 18756 raft_consensus.cc:729] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e1f2b0ba4aa42298262543d325ea10b, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.929886 18756 consensus_queue.cc:260] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.929962 18756 ts_tablet_manager.cc:1428] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:34.930246 18233 catalog_manager.cc:4841] TS 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195:39997): tablet 65453353fe924306a4154ccddcab5a67 (table test-table [id=c794f7e39e5b420f84f1d08ed5c94947]) successfully deleted
I20260924 07:14:34.930439 18759 tablet_bootstrap.cc:492] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70: Bootstrap starting.
I20260924 07:14:34.930682 18233 catalog_manager.cc:4841] TS 54ff4dd74c094d099b72951152543b70 (127.10.130.194:37809): tablet 65453353fe924306a4154ccddcab5a67 (table test-table [id=c794f7e39e5b420f84f1d08ed5c94947]) successfully deleted
I20260924 07:14:34.930751 18759 tablet_bootstrap.cc:654] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:34.930887 18755 tablet_bootstrap.cc:492] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2: No bootstrap required, opened a new log
I20260924 07:14:34.930934 18755 ts_tablet_manager.cc:1397] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:34.931051 18755 raft_consensus.cc:348] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.931100 18755 raft_consensus.cc:374] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.931118 18755 raft_consensus.cc:729] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b37ca7d22c24e45a9f9abddf38779d2, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.931154 18755 consensus_queue.cc:260] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.931238 18759 tablet_bootstrap.cc:492] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70: No bootstrap required, opened a new log
I20260924 07:14:34.931238 18755 ts_tablet_manager.cc:1428] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:34.931277 18759 ts_tablet_manager.cc:1397] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:34.931360 18759 raft_consensus.cc:348] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.931419 18759 raft_consensus.cc:374] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:34.931445 18759 raft_consensus.cc:729] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ff4dd74c094d099b72951152543b70, State: Initialized, Role: FOLLOWER
I20260924 07:14:34.931483 18759 consensus_queue.cc:260] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.931561 18759 ts_tablet_manager.cc:1428] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:34.980681 18761 raft_consensus.cc:482] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:34.980777 18761 raft_consensus.cc:504] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.981096 18761 leader_election.cc:288] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193:46589), 54ff4dd74c094d099b72951152543b70 (127.10.130.194:37809)
I20260924 07:14:34.983937 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.981525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:52045 (local address 127.10.130.194:37809)
0924 07:14:34.981572 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.981574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.981581 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:14:34.981634 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.981635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.981644 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.981688 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:34.981822 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.982280 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.983294 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.983635 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.983693 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.983702 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.983840 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.983871 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:14:34.983943 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.981327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:37457 (local address 127.10.130.193:46589)
0924 07:14:34.981378 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:34.981381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:34.981444 (+ 63us) server_negotiation.cc:407] Connection header received
0924 07:14:34.981557 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:34.981558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:34.981574 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:34.981629 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:34.981820 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.982282 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.983290 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.983637 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.983698 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.983706 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:34.983837 (+ 131us) server_negotiation.cc:299] Negotiation successful
0924 07:14:34.983878 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:14:34.983969 18834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.981265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:46589 (local address 127.10.130.195:37457)
0924 07:14:34.981371 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.981383 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.981448 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.981639 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.981640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.981643 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:34.981758 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.981760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.982368 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.982369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.983208 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.983211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.983705 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.983706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.983777 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.983781 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.983826 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.983866 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":68,"thread_start_us":52,"threads_started":1}
I20260924 07:14:34.983994 18835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:34.981423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:37809 (local address 127.10.130.195:52045)
0924 07:14:34.981505 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:34.981515 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:34.981572 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:34.981699 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:34.981700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:34.981702 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:34.981766 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.981767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.982385 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.982386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.983212 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:34.983214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:34.983707 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:34.983708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:34.983779 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:34.983782 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:34.983828 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 07:14:34.983869 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":36,"thread_start_us":43,"threads_started":1}
I20260924 07:14:34.984217 18404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ddd9ac4d21442c3af246bff1bceaff7" candidate_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e1f2b0ba4aa42298262543d325ea10b" is_pre_election: true
I20260924 07:14:34.984318 18404 raft_consensus.cc:2454] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b37ca7d22c24e45a9f9abddf38779d2 in term 0.
I20260924 07:14:34.984488 18617 leader_election.cc:302] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b37ca7d22c24e45a9f9abddf38779d2, 6e1f2b0ba4aa42298262543d325ea10b; no voters:
I20260924 07:14:34.984558 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ddd9ac4d21442c3af246bff1bceaff7" candidate_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ff4dd74c094d099b72951152543b70" is_pre_election: true
I20260924 07:14:34.984614 18761 raft_consensus.cc:2781] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:34.984640 18761 raft_consensus.cc:482] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:34.984645 18544 raft_consensus.cc:2454] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b37ca7d22c24e45a9f9abddf38779d2 in term 0.
I20260924 07:14:34.984658 18761 raft_consensus.cc:3037] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.985357 18761 raft_consensus.cc:504] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.985459 18761 leader_election.cc:288] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [CANDIDATE]: Term 1 election: Requested vote from peers 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193:46589), 54ff4dd74c094d099b72951152543b70 (127.10.130.194:37809)
I20260924 07:14:34.985633 18404 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ddd9ac4d21442c3af246bff1bceaff7" candidate_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e1f2b0ba4aa42298262543d325ea10b"
I20260924 07:14:34.985646 18544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ddd9ac4d21442c3af246bff1bceaff7" candidate_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54ff4dd74c094d099b72951152543b70"
I20260924 07:14:34.985708 18404 raft_consensus.cc:3037] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.985721 18544 raft_consensus.cc:3037] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:34.986302 18404 raft_consensus.cc:2454] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b37ca7d22c24e45a9f9abddf38779d2 in term 1.
I20260924 07:14:34.986328 18544 raft_consensus.cc:2454] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b37ca7d22c24e45a9f9abddf38779d2 in term 1.
I20260924 07:14:34.986482 18617 leader_election.cc:302] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b37ca7d22c24e45a9f9abddf38779d2, 54ff4dd74c094d099b72951152543b70; no voters:
I20260924 07:14:34.986565 18761 raft_consensus.cc:2781] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:34.986672 18761 raft_consensus.cc:686] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 LEADER]: Becoming Leader. State: Replica: 0b37ca7d22c24e45a9f9abddf38779d2, State: Running, Role: LEADER
I20260924 07:14:34.986747 18761 consensus_queue.cc:237] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } }
I20260924 07:14:34.987377 18245 catalog_manager.cc:5495] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195). New cstate: current_term: 1 leader_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b37ca7d22c24e45a9f9abddf38779d2" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39997 } health_report { overall_health: HEALTHY } } }
I20260924 07:14:35.006047 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:35.003842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57622 (local address 127.10.130.195:39997)
0924 07:14:35.003888 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:35.003889 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:35.003897 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:35.003960 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:35.003961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:35.004003 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:35.004066 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:35.004192 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:35.004690 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:35.005371 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:35.005560 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:35.005610 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:35.005751 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:35.005848 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:35.005908 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:35.005963 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 07:14:35.005978 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 07:14:35.009919 18750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:35.008077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:39801 (local address 127.10.130.193:46589)
0924 07:14:35.008116 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:35.008118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:35.008204 (+ 86us) server_negotiation.cc:407] Connection header received
0924 07:14:35.008239 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:35.008240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:35.008248 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:35.008289 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:35.008378 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:35.008806 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:35.009450 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:35.009613 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:35.009651 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:35.009776 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:35.009798 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:35.009828 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:35.009858 (+ 30us) server_negotiation.cc:299] Negotiation successful
0924 07:14:35.009871 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:14:35.009950 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:35.008025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:46589 (local address 127.10.130.195:39801)
0924 07:14:35.008145 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:35.008155 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:35.008205 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:35.008290 (+ 85us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:35.008291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:35.008292 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 07:14:35.008340 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:35.008341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:35.008851 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:35.008851 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:35.009398 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:35.009399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:35.009672 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:35.009672 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:35.009710 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:35.009721 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 07:14:35.009828 (+ 107us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 07:14:35.009830 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:14:35.009857 (+ 27us) client_negotiation.cc:240] Negotiation successful
0924 07:14:35.009884 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":52,"threads_started":1}
I20260924 07:14:35.011693 18544 raft_consensus.cc:1261] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Refusing update from remote peer 0b37ca7d22c24e45a9f9abddf38779d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:35.011698 18404 raft_consensus.cc:1261] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Refusing update from remote peer 0b37ca7d22c24e45a9f9abddf38779d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:35.011951 18836 consensus_queue.cc:1035] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54ff4dd74c094d099b72951152543b70" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 37809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:35.012096 18761 consensus_queue.cc:1035] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e1f2b0ba4aa42298262543d325ea10b" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:35.018389 18753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:35.016489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:39865 (local address 127.10.130.195:39997)
0924 07:14:35.016527 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:35.016528 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:35.016534 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:14:35.016548 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:35.016549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:35.016556 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:35.016592 (+ 36us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:35.016659 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:35.017079 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:35.017702 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:35.017956 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:35.018005 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:35.018133 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:35.018161 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:35.018210 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:35.018305 (+ 95us) server_negotiation.cc:299] Negotiation successful
0924 07:14:35.018323 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:14:35.018409 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:35.016354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:39997 (local address 127.10.130.193:39865)
0924 07:14:35.016393 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:35.016403 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:35.016440 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:35.016595 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:35.016596 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:35.016597 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 07:14:35.016627 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:35.016628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:35.017133 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:35.017134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:35.017645 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:35.017646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:35.018010 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:35.018011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:35.018063 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:35.018069 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 07:14:35.018249 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 07:14:35.018251 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:14:35.018292 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 07:14:35.018333 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 07:14:35.024554 18246 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:35.024632 18246 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:35.025143 18246 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ad253ca82294b5fae64f46deb86ce1c: Sending DeleteTablet for 3 replicas of tablet 6ddd9ac4d21442c3af246bff1bceaff7
I20260924 07:14:35.025341 18384 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ddd9ac4d21442c3af246bff1bceaff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:35 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:59668
I20260924 07:14:35.025426 18524 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ddd9ac4d21442c3af246bff1bceaff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:35 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:45488
I20260924 07:14:35.025460 18664 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ddd9ac4d21442c3af246bff1bceaff7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:35 UTC) from {username='oryx', principal='oryx/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:57618
I20260924 07:14:35.025506 18778 tablet_replica.cc:330] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b: stopping tablet replica
I20260924 07:14:35.025539 18779 tablet_replica.cc:330] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2: stopping tablet replica
I20260924 07:14:35.025549 18778 raft_consensus.cc:2229] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:35.025579 18778 raft_consensus.cc:2258] T 6ddd9ac4d21442c3af246bff1bceaff7 P 6e1f2b0ba4aa42298262543d325ea10b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:35.025581 18779 raft_consensus.cc:2229] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:35.025646 18779 raft_consensus.cc:2258] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:35.025767 18780 tablet_replica.cc:330] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70: stopping tablet replica
I20260924 07:14:35.025815 18780 raft_consensus.cc:2229] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:35.025815 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 18319
I20260924 07:14:35.025842 18780 raft_consensus.cc:2258] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:35.026947 18780 ts_tablet_manager.cc:1905] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:35.028153 18779 ts_tablet_manager.cc:1905] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:35.028518 18780 ts_tablet_manager.cc:1918] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:35.028565 18780 log.cc:1196] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/6ddd9ac4d21442c3af246bff1bceaff7
I20260924 07:14:35.028759 18780 ts_tablet_manager.cc:1939] T 6ddd9ac4d21442c3af246bff1bceaff7 P 54ff4dd74c094d099b72951152543b70: Deleting consensus metadata
I20260924 07:14:35.029155 18779 ts_tablet_manager.cc:1918] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 07:14:35.029206 18779 log.cc:1196] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/6ddd9ac4d21442c3af246bff1bceaff7
I20260924 07:14:35.029196 18233 catalog_manager.cc:4841] TS 54ff4dd74c094d099b72951152543b70 (127.10.130.194:37809): tablet 6ddd9ac4d21442c3af246bff1bceaff7 (table test-table [id=937085ca085d4f5e92b41de9f849e264]) successfully deleted
I20260924 07:14:35.029371 18779 ts_tablet_manager.cc:1939] T 6ddd9ac4d21442c3af246bff1bceaff7 P 0b37ca7d22c24e45a9f9abddf38779d2: Deleting consensus metadata
I20260924 07:14:35.029856 18233 catalog_manager.cc:4841] TS 0b37ca7d22c24e45a9f9abddf38779d2 (127.10.130.195:39997): tablet 6ddd9ac4d21442c3af246bff1bceaff7 (table test-table [id=937085ca085d4f5e92b41de9f849e264]) successfully deleted
W20260924 07:14:35.033557 18233 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:46589 (error 108)
I20260924 07:14:35.033829 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 18459
I20260924 07:14:35.037669 18846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:35.033931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:46589 (local address 127.0.0.1:59684)
0924 07:14:35.037583 (+ 3652us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:35.037630 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:46589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3635,"thread_start_us":47,"threads_started":1}
W20260924 07:14:35.037757 18233 catalog_manager.cc:4584] TS 6e1f2b0ba4aa42298262543d325ea10b (127.10.130.193:46589): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6ddd9ac4d21442c3af246bff1bceaff7: Network error: Client connection negotiation failed: client connection to 127.10.130.193:46589: connect: Connection refused (error 111)
I20260924 07:14:35.040796 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 18599
I20260924 07:14:35.044723 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 18215
2026-09-24T07:14:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2609 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:35 dist-test-slave-7q7m krb5kdc[18849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:35 dist-test-slave-7q7m krb5kdc[18849](info): set up 2 sockets
Sep 24 07:14:35 dist-test-slave-7q7m krb5kdc[18849](info): commencing operation
krb5kdc: starting...
W20260924 07:14:35.988153 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:36 dist-test-slave-7q7m krb5kdc[18849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234076, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:14:36.091398 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:46217
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:36301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:46217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:36.167482 18862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:36.167657 18862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:36.167676 18862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:36.168741 18862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:36.168774 18862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:36.168787 18862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:36.168798 18862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:36.168812 18862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:36.170140 18862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:36301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:46217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:46217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:36.170315 18862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:36.170468 18862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:36.174674 18868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:36.174757 18862 server_base.cc:1034] running on GCE node
W20260924 07:14:36.174682 18870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:36.174858 18867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:36.175077 18862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:36.175364 18862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:36.176493 18862 hybrid_clock.cc:648] HybridClock initialized: now 1790234076176473 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-24T07:14:36Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1134 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:36 dist-test-slave-7q7m krb5kdc[18875](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:36 dist-test-slave-7q7m krb5kdc[18875](info): set up 2 sockets
Sep 24 07:14:36 dist-test-slave-7q7m krb5kdc[18875](info): commencing operation
krb5kdc: starting...
W20260924 07:14:37.116098 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:37 dist-test-slave-7q7m krb5kdc[18875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:14:37.216717 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:34947
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:34947
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:37.294682 18888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:37.294863 18888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:37.294891 18888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:37.295956 18888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:37.296010 18888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:37.296025 18888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:37.296036 18888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:37.296052 18888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:37.297478 18888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:34947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:34947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:37.297658 18888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:37.297813 18888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:37.302161 18894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:37.302167 18896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:37.302192 18888 server_base.cc:1034] running on GCE node
W20260924 07:14:37.302174 18893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:37.302582 18888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:37.302846 18888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:37.304006 18888 hybrid_clock.cc:648] HybridClock initialized: now 1790234077303988 us; error 29 us; skew 500 ppm
Sep 24 07:14:37 dist-test-slave-7q7m krb5kdc[18875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234077, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:37.306182 18888 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:14:37.306562 18888 webserver.cc:460] Webserver started at http://127.10.130.254:33959/ using document root <none> and password file <none>
I20260924 07:14:37.306751 18888 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:37.306798 18888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:37.306916 18888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:37.307858 18888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "988535aef2e64c05959ba91a3786dc2b"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "6ecf1fbd817dd0363c604c73af8b62d7"
server_key_iv: "c87564839eb7141c7b50cc0d06cd0387"
server_key_version: "encryptionkey@0"
I20260924 07:14:37.308194 18888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "988535aef2e64c05959ba91a3786dc2b"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "6ecf1fbd817dd0363c604c73af8b62d7"
server_key_iv: "c87564839eb7141c7b50cc0d06cd0387"
server_key_version: "encryptionkey@0"
I20260924 07:14:37.309320 18888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:37.309929 18903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:37.310070 18888 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:37.310146 18888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "988535aef2e64c05959ba91a3786dc2b"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "6ecf1fbd817dd0363c604c73af8b62d7"
server_key_iv: "c87564839eb7141c7b50cc0d06cd0387"
server_key_version: "encryptionkey@0"
I20260924 07:14:37.310201 18888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:37.327754 18888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:37.329483 18888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:37.329620 18888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:37.333216 18888 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:34947
I20260924 07:14:37.333252 18955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:34947 every 8 connection(s)
I20260924 07:14:37.333567 18888 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:37.334344 18956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:37.336593 18956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b: Bootstrap starting.
I20260924 07:14:37.337137 18956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:37.337386 18956 log.cc:824] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:37.338107 18956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b: No bootstrap required, opened a new log
I20260924 07:14:37.339690 18956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988535aef2e64c05959ba91a3786dc2b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34947 } }
I20260924 07:14:37.339859 18956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:37.339883 18956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 988535aef2e64c05959ba91a3786dc2b, State: Initialized, Role: FOLLOWER
I20260924 07:14:37.340057 18956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988535aef2e64c05959ba91a3786dc2b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34947 } }
I20260924 07:14:37.340133 18956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:37.340159 18956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:37.340197 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 18888
I20260924 07:14:37.340211 18956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:37.340309 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:37.340485 10763 external_mini_cluster.cc:1428] Setting key 44e53597ab57fa1c164a665985a148fd
I20260924 07:14:37.340849 18956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988535aef2e64c05959ba91a3786dc2b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34947 } }
I20260924 07:14:37.340966 18956 leader_election.cc:302] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 988535aef2e64c05959ba91a3786dc2b; no voters:
I20260924 07:14:37.341179 18956 leader_election.cc:288] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:37.341213 18961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:37.341361 18961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 1 LEADER]: Becoming Leader. State: Replica: 988535aef2e64c05959ba91a3786dc2b, State: Running, Role: LEADER
I20260924 07:14:37.341434 18956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:37.341497 18961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988535aef2e64c05959ba91a3786dc2b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34947 } }
I20260924 07:14:37.341836 18961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 988535aef2e64c05959ba91a3786dc2b. Latest consensus state: current_term: 1 leader_uuid: "988535aef2e64c05959ba91a3786dc2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988535aef2e64c05959ba91a3786dc2b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34947 } } }
I20260924 07:14:37.341907 18961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:37.342047 18963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "988535aef2e64c05959ba91a3786dc2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988535aef2e64c05959ba91a3786dc2b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34947 } } }
I20260924 07:14:37.342101 18963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:37.342335 18965 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:37.342849 18965 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:37.344492 18965 catalog_manager.cc:1347] Generated new cluster ID: 7420fc2f78664714afdf44ec366a3295
I20260924 07:14:37.344549 18965 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:14:37 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:37.350054 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:37.340918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.10.130.254:34947)
0924 07:14:37.341068 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:37.341078 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:37.341113 (+ 35us) server_negotiation.cc:407] Connection header received
0924 07:14:37.341288 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:37.341297 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:37.341359 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:37.342184 (+ 825us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:37.342348 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.343718 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.344709 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.345112 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.345247 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:37.347222 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:37.347227 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:37.347227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:37.347238 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:37.348750 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:37.349052 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:37.349055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:37.349055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:37.349150 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:37.349324 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:37.349325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:37.349325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:37.349539 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:37.349682 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:37.349782 (+ 100us) server_negotiation.cc:299] Negotiation successful
0924 07:14:37.349912 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260924 07:14:37.350762 18965 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:37.351388 18965 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:37.356015 18965 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b: Generated new TSK 0
I20260924 07:14:37.356177 18965 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:14:37.380820 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:34947
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:37.457888 18982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:37.458068 18982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:37.458097 18982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:37.459189 18982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:37.459225 18982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:37.459260 18982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:37.460671 18982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:34947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:37.460886 18982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:37.461068 18982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:37.465615 18988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:37.465720 18990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:37.465611 18987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:37.465962 18982 server_base.cc:1034] running on GCE node
I20260924 07:14:37.466146 18982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:37.466352 18982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:37.467489 18982 hybrid_clock.cc:648] HybridClock initialized: now 1790234077467476 us; error 30 us; skew 500 ppm
Sep 24 07:14:37 dist-test-slave-7q7m krb5kdc[18875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234077, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:37.469708 18982 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:14:37.470117 18982 webserver.cc:460] Webserver started at http://127.10.130.193:46585/ using document root <none> and password file <none>
I20260924 07:14:37.470309 18982 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:37.470350 18982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:37.470458 18982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:37.471443 18982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "7e1aa5d45302411e9db59e844c5d94da"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "93193673fa2bb283a0919c21d0d2dc39"
server_key_iv: "b183a277f842cc2bf46c2cbac5bb0799"
server_key_version: "encryptionkey@0"
I20260924 07:14:37.471755 18982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "7e1aa5d45302411e9db59e844c5d94da"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "93193673fa2bb283a0919c21d0d2dc39"
server_key_iv: "b183a277f842cc2bf46c2cbac5bb0799"
server_key_version: "encryptionkey@0"
I20260924 07:14:37.472970 18982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 07:14:37.473600 18997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:37.473738 18982 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:37.473807 18982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "7e1aa5d45302411e9db59e844c5d94da"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "93193673fa2bb283a0919c21d0d2dc39"
server_key_iv: "b183a277f842cc2bf46c2cbac5bb0799"
server_key_version: "encryptionkey@0"
I20260924 07:14:37.473872 18982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:37.480402 18982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:37.482261 18982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:37.482376 18982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:37.482553 18982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:37.482797 18982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:37.482826 18982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:37.482846 18982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:37.482870 18982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:37.488060 18982 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:42649
I20260924 07:14:37.488116 19110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:42649 every 8 connection(s)
I20260924 07:14:37.488380 18982 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
Sep 24 07:14:37 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234077, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:37.494800 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 18982
I20260924 07:14:37.494894 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:37.495057 10763 external_mini_cluster.cc:1428] Setting key b9331c59d00198a98abbb60bfaf8f613
I20260924 07:14:37.496168 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:37.488967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:48381 (local address 127.10.130.254:34947)
0924 07:14:37.489013 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:37.489015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:37.489182 (+ 167us) server_negotiation.cc:407] Connection header received
0924 07:14:37.489602 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:37.489603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:37.489615 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:37.489648 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:37.490446 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.490925 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.491785 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.491963 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.492046 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:37.493722 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:37.493724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:37.493724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:37.493730 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:37.494894 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:37.495275 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:37.495276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:37.495277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:37.495337 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:37.495585 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:37.495585 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:37.495586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:37.495711 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:37.495777 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:37.496011 (+ 234us) server_negotiation.cc:299] Negotiation successful
0924 07:14:37.496084 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:14:37.496296 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:37.488951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:48381)
0924 07:14:37.489077 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:37.489107 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:37.489523 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:37.489703 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:37.489706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:37.490209 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:37.490379 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:37.490383 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.490986 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.490986 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:37.491730 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:37.491731 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.492051 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.492052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:37.492144 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:37.492473 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:37.492480 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:37.493649 (+ 1169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:37.494965 (+ 1316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:37.494972 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:37.494974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:37.495203 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:37.495403 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:37.495404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:37.495404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:37.495528 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:37.495768 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:37.495772 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:37.495875 (+ 103us) client_negotiation.cc:769] Sending connection context
0924 07:14:37.495948 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 07:14:37.496125 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":71,"threads_started":1}
I20260924 07:14:37.496838 19111 heartbeater.cc:347] Connected to a master server at 127.10.130.254:34947
I20260924 07:14:37.496929 19111 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:37.497109 19111 heartbeater.cc:511] Master 127.10.130.254:34947 requested a full tablet report, sending...
I20260924 07:14:37.497604 18920 ts_manager.cc:194] Registered new tserver with Master: 7e1aa5d45302411e9db59e844c5d94da (127.10.130.193:42649)
I20260924 07:14:37.498337 18920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:48381
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:14:37.516280 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:34947
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:37.593909 19116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:37.594094 19116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:37.594115 19116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:37.595204 19116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:37.595237 19116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:37.595261 19116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:37.596660 19116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:34947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:37.596838 19116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:37.597008 19116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 07:14:37.601742 19116 server_base.cc:1034] running on GCE node
W20260924 07:14:37.601684 19122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:37.601670 19124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:37.601697 19121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:37.602036 19116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:37.602246 19116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:37.603377 19116 hybrid_clock.cc:648] HybridClock initialized: now 1790234077603359 us; error 34 us; skew 500 ppm
Sep 24 07:14:37 dist-test-slave-7q7m krb5kdc[18875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234077, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:37.605700 19116 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:14:37.606129 19116 webserver.cc:460] Webserver started at http://127.10.130.194:42331/ using document root <none> and password file <none>
I20260924 07:14:37.606334 19116 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:37.606380 19116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:37.606510 19116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:37.607537 19116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "7eba6b34aad1487399b2fef8f3b289cc"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "a1eddeddb23215a86ade28b890653913"
server_key_iv: "8dadc1701ca1af550fde785c9aecd4a7"
server_key_version: "encryptionkey@0"
I20260924 07:14:37.607844 19116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "7eba6b34aad1487399b2fef8f3b289cc"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "a1eddeddb23215a86ade28b890653913"
server_key_iv: "8dadc1701ca1af550fde785c9aecd4a7"
server_key_version: "encryptionkey@0"
I20260924 07:14:37.609076 19116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 07:14:37.609761 19131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:37.609915 19116 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:37.609994 19116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "7eba6b34aad1487399b2fef8f3b289cc"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "a1eddeddb23215a86ade28b890653913"
server_key_iv: "8dadc1701ca1af550fde785c9aecd4a7"
server_key_version: "encryptionkey@0"
I20260924 07:14:37.610057 19116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:37.623811 19116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:37.625554 19116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:37.625686 19116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:37.625886 19116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:37.626163 19116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:37.626195 19116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:37.626224 19116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:37.626247 19116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:37.631964 19116 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:46701
I20260924 07:14:37.632036 19244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:46701 every 8 connection(s)
I20260924 07:14:37.632329 19116 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
Sep 24 07:14:37 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234077, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:37.640194 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 19116
I20260924 07:14:37.640292 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:37.640497 10763 external_mini_cluster.cc:1428] Setting key 8bc7f4f798183f8240f40292ba4f1339
I20260924 07:14:37.640568 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:37.633038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:49055 (local address 127.10.130.254:34947)
0924 07:14:37.633102 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:37.633104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:37.633269 (+ 165us) server_negotiation.cc:407] Connection header received
0924 07:14:37.633732 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:37.633733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:37.633745 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:37.633811 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:37.634691 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.635182 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.636152 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.636341 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.636392 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:37.638253 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:37.638255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:37.638255 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:37.638262 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:37.639451 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:37.639748 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:37.639748 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:37.639749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:37.639792 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:37.639959 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:37.639960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:37.639960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:37.640104 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:37.640171 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:37.640400 (+ 229us) server_negotiation.cc:299] Negotiation successful
0924 07:14:37.640481 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 07:14:37.640815 19247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:37.632997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:49055)
0924 07:14:37.633145 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:37.633185 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:37.633655 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:37.633868 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:37.633871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:37.634410 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:37.634610 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:37.634615 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.635251 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.635251 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:37.636087 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:37.636090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.636408 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.636409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:37.636503 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:37.636924 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:37.636933 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:37.638176 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:37.639508 (+ 1332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:37.639511 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:37.639512 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:37.639696 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:37.639835 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:37.639835 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:37.639835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:37.639920 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:37.640160 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:37.640163 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:37.640276 (+ 113us) client_negotiation.cc:769] Sending connection context
0924 07:14:37.640375 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 07:14:37.640508 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260924 07:14:37.641245 19245 heartbeater.cc:347] Connected to a master server at 127.10.130.254:34947
I20260924 07:14:37.641338 19245 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:37.641513 19245 heartbeater.cc:511] Master 127.10.130.254:34947 requested a full tablet report, sending...
I20260924 07:14:37.641914 18920 ts_manager.cc:194] Registered new tserver with Master: 7eba6b34aad1487399b2fef8f3b289cc (127.10.130.194:46701)
I20260924 07:14:37.642318 18920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:49055
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:14:37.662590 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:34947
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:37.739260 19250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:37.739442 19250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:37.739463 19250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:37.740546 19250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:37.740581 19250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:37.740604 19250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:37.742038 19250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:34947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:37.742215 19250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:37.742400 19250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:37.746833 19256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:37.746863 19258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:37.746843 19255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:37.747033 19250 server_base.cc:1034] running on GCE node
I20260924 07:14:37.747231 19250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:37.747406 19250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:37.748529 19250 hybrid_clock.cc:648] HybridClock initialized: now 1790234077748517 us; error 27 us; skew 500 ppm
Sep 24 07:14:37 dist-test-slave-7q7m krb5kdc[18875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234077, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:37.750758 19250 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:14:37.751169 19250 webserver.cc:460] Webserver started at http://127.10.130.195:42375/ using document root <none> and password file <none>
I20260924 07:14:37.751349 19250 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:37.751392 19250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:37.751514 19250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:37.752503 19250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "1990b10887744909a20d0831a8794094"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "84eccdb24f344a655629cc538ef49489"
server_key_iv: "fba6033621c4c9d84fc1c89d3cf60875"
server_key_version: "encryptionkey@0"
I20260924 07:14:37.752801 19250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "1990b10887744909a20d0831a8794094"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "84eccdb24f344a655629cc538ef49489"
server_key_iv: "fba6033621c4c9d84fc1c89d3cf60875"
server_key_version: "encryptionkey@0"
I20260924 07:14:37.753939 19250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:37.754528 19265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:37.754668 19250 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:37.754737 19250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "1990b10887744909a20d0831a8794094"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "84eccdb24f344a655629cc538ef49489"
server_key_iv: "fba6033621c4c9d84fc1c89d3cf60875"
server_key_version: "encryptionkey@0"
I20260924 07:14:37.754797 19250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:37.765972 19250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:37.767711 19250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:37.767839 19250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:37.768065 19250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:37.768386 19250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:37.768427 19250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:37.768455 19250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:37.768471 19250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:37.773808 19250 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:39799
I20260924 07:14:37.773851 19378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:39799 every 8 connection(s)
I20260924 07:14:37.774148 19250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:37.776774 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 19250
I20260924 07:14:37.776880 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:37.777056 10763 external_mini_cluster.cc:1428] Setting key aec6e798651e604f7c03e679a4debea3
Sep 24 07:14:37 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234077, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:37.782977 18960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:37.774784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:58543 (local address 127.10.130.254:34947)
0924 07:14:37.774860 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:37.774862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:37.775044 (+ 182us) server_negotiation.cc:407] Connection header received
0924 07:14:37.775540 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:37.775541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:37.775553 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:37.775604 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:37.776589 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.777196 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.778306 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.778507 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.778560 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:37.780586 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:37.780587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:37.780587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:37.780595 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:37.781805 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:37.782147 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:37.782148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:37.782148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:37.782199 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:37.782404 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:37.782405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:37.782406 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:37.782513 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:37.782566 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:37.782815 (+ 249us) server_negotiation.cc:299] Negotiation successful
0924 07:14:37.782889 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 07:14:37.783146 19381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:37.774746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:58543)
0924 07:14:37.774933 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:37.774968 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:37.775448 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:37.775675 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:37.775679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:37.776277 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:37.776505 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:37.776511 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.777272 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.777273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:37.778234 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:37.778237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:37.778570 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:37.778571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:37.778686 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:37.779144 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:37.779152 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:37.780499 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:37.781876 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:37.781882 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:37.781884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:37.782085 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:37.782253 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:37.782253 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:37.782254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:37.782353 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:37.782564 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:37.782569 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:37.782698 (+ 129us) client_negotiation.cc:769] Sending connection context
0924 07:14:37.782787 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 07:14:37.782938 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":79,"threads_started":1}
I20260924 07:14:37.783602 19379 heartbeater.cc:347] Connected to a master server at 127.10.130.254:34947
I20260924 07:14:37.783675 19379 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:37.783815 19379 heartbeater.cc:511] Master 127.10.130.254:34947 requested a full tablet report, sending...
I20260924 07:14:37.784124 18920 ts_manager.cc:194] Registered new tserver with Master: 1990b10887744909a20d0831a8794094 (127.10.130.195:39799)
I20260924 07:14:37.784489 18920 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:58543
I20260924 07:14:37.788663 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab.
Entry for principal oryx/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab.
WARNING: no policy specified for oryx/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab.
Entry for principal oryx/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab.
WARNING: no policy specified for oryx/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab.
Entry for principal oryx/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab.
I20260924 07:14:37.849401 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 18982
I20260924 07:14:37.853214 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 19116
I20260924 07:14:37.857241 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 19250
I20260924 07:14:37.861037 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 18888
I20260924 07:14:37.865020 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:34947
--webserver_interface=127.10.130.254
--webserver_port=33959
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:34947
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:37.941985 19388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:37.942171 19388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:37.942191 19388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:37.943300 19388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:37.943338 19388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:37.943351 19388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:37.943361 19388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:37.943375 19388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:37.944825 19388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:34947
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:34947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=33959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:37.945055 19388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:37.945250 19388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:37.949594 19393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:37.949627 19394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:37.949632 19396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:37.949739 19388 server_base.cc:1034] running on GCE node
I20260924 07:14:37.950019 19388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:37.950256 19388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:37.951396 19388 hybrid_clock.cc:648] HybridClock initialized: now 1790234077951374 us; error 29 us; skew 500 ppm
Sep 24 07:14:37 dist-test-slave-7q7m krb5kdc[18875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234077, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:37.953656 19388 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:14:37.954078 19388 webserver.cc:460] Webserver started at http://127.10.130.254:33959/ using document root <none> and password file <none>
I20260924 07:14:37.954267 19388 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:37.954325 19388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:37.955581 19388 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:37.956175 19403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:37.956307 19388 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:37.956385 19388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "988535aef2e64c05959ba91a3786dc2b"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "6ecf1fbd817dd0363c604c73af8b62d7"
server_key_iv: "c87564839eb7141c7b50cc0d06cd0387"
server_key_version: "encryptionkey@0"
I20260924 07:14:37.956705 19388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:37.963912 19388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:37.965845 19388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:37.965962 19388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:37.969466 19388 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:34947
I20260924 07:14:37.969496 19455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:34947 every 8 connection(s)
I20260924 07:14:37.969812 19388 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:37.971091 19456 sys_catalog.cc:263] Verifying existing consensus state
I20260924 07:14:37.971639 19456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b: Bootstrap starting.
I20260924 07:14:37.978917 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 19388
I20260924 07:14:37.979074 19456 log.cc:824] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:37.979233 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:42649
--local_ip_for_outbound_sockets=127.10.130.193
--tserver_master_addrs=127.10.130.254:34947
--webserver_port=46585
--webserver_interface=127.10.130.193
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.130.193
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:14:37.980934 19456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 07:14:37.981221 19456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b: Bootstrap complete.
I20260924 07:14:37.983214 19456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988535aef2e64c05959ba91a3786dc2b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34947 } }
I20260924 07:14:37.983433 19456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 988535aef2e64c05959ba91a3786dc2b, State: Initialized, Role: FOLLOWER
I20260924 07:14:37.983585 19456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988535aef2e64c05959ba91a3786dc2b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34947 } }
I20260924 07:14:37.983660 19456 raft_consensus.cc:388] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:37.983704 19456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:37.983749 19456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 1 FOLLOWER]: Advancing to term 2
I20260924 07:14:37.984747 19456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988535aef2e64c05959ba91a3786dc2b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34947 } }
I20260924 07:14:37.984838 19456 leader_election.cc:302] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 988535aef2e64c05959ba91a3786dc2b; no voters:
I20260924 07:14:37.984990 19456 leader_election.cc:288] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 07:14:37.985038 19460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 2 FOLLOWER]: Leader election won for term 2
I20260924 07:14:37.985152 19460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [term 2 LEADER]: Becoming Leader. State: Replica: 988535aef2e64c05959ba91a3786dc2b, State: Running, Role: LEADER
I20260924 07:14:37.985245 19456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:37.985241 19460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988535aef2e64c05959ba91a3786dc2b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34947 } }
I20260924 07:14:37.985533 19460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 988535aef2e64c05959ba91a3786dc2b. Latest consensus state: current_term: 2 leader_uuid: "988535aef2e64c05959ba91a3786dc2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988535aef2e64c05959ba91a3786dc2b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34947 } } }
I20260924 07:14:37.985601 19460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:37.985728 19461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "988535aef2e64c05959ba91a3786dc2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "988535aef2e64c05959ba91a3786dc2b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34947 } } }
I20260924 07:14:37.985790 19461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:37.986044 19465 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:37.986522 19465 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:37.986645 19465 catalog_manager.cc:1259] Loaded cluster ID: 7420fc2f78664714afdf44ec366a3295
I20260924 07:14:37.986690 19465 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:37.987536 19465 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:37.987679 19465 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 988535aef2e64c05959ba91a3786dc2b: Loaded TSK: 0
I20260924 07:14:37.987870 19465 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 07:14:38.057607 19459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:38.057799 19459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:38.057828 19459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:38.058964 19459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:38.059001 19459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:38.059034 19459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:38.060487 19459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.193.keytab
--principal=oryx/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:42649
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=46585
--tserver_master_addrs=127.10.130.254:34947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:38.060698 19459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:38.060889 19459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:38.065742 19482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:38.065747 19485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:38.065747 19483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:38.066062 19459 server_base.cc:1034] running on GCE node
I20260924 07:14:38.066339 19459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:38.066638 19459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:38.067798 19459 hybrid_clock.cc:648] HybridClock initialized: now 1790234078067787 us; error 27 us; skew 500 ppm
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234078, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:38.070534 19459 init.cc:377] Logged in from keytab as oryx/127.10.130.193@KRBTEST.COM (short username oryx)
I20260924 07:14:38.070959 19459 webserver.cc:460] Webserver started at http://127.10.130.193:46585/ using document root <none> and password file <none>
I20260924 07:14:38.071170 19459 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:38.071237 19459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:38.072557 19459 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:38.073127 19492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:38.073305 19459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:38.073385 19459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "7e1aa5d45302411e9db59e844c5d94da"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "93193673fa2bb283a0919c21d0d2dc39"
server_key_iv: "b183a277f842cc2bf46c2cbac5bb0799"
server_key_version: "encryptionkey@0"
I20260924 07:14:38.073715 19459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:38.083639 19459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:38.085664 19459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:38.085815 19459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:38.086004 19459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:38.086267 19459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:38.086298 19459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:38.086326 19459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:38.086349 19459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:38.091609 19459 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:42649
I20260924 07:14:38.091706 19605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:42649 every 8 connection(s)
I20260924 07:14:38.092025 19459 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:14:38.093801 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 19459
I20260924 07:14:38.094099 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:46701
--local_ip_for_outbound_sockets=127.10.130.194
--tserver_master_addrs=127.10.130.254:34947
--webserver_port=42331
--webserver_interface=127.10.130.194
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.130.194
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:38.102592 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:38.092710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:41257)
0924 07:14:38.092908 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:38.092953 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:38.093531 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:38.094136 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:38.094274 (+ 138us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:38.094977 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:38.095250 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.095257 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.096927 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.096928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.098192 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.098195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.099045 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.099046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.099191 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.099762 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:38.099772 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:38.102406 (+ 2634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
W20260924 07:14:38.102797 19606 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:34947 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.130.254:34947: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
W20260924 07:14:38.103058 19609 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:38.092803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:41257 (local address 127.10.130.254:34947)
0924 07:14:38.092992 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:38.092999 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:38.093025 (+ 26us) server_negotiation.cc:407] Connection header received
0924 07:14:38.093735 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:38.093743 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:38.093802 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:38.093983 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:38.095383 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.096818 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.098283 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.098950 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.099053 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.102559 (+ 3506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:41257: BlockingRecv error: recv got EOF from 127.10.130.193:41257 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":76,"threads_started":1}
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:38.108307 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:38.102986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:51383)
0924 07:14:38.103255 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:38.103264 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:38.103301 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:38.103506 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:38.103507 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:38.103643 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:38.103691 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.103692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.104631 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.104632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.105464 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.105466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.105901 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.105902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.105960 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.106363 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:38.106368 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:38.108238 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":59,"threads_started":1}
W20260924 07:14:38.108784 19609 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:38.103126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:51383 (local address 127.10.130.254:34947)
0924 07:14:38.103388 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:38.103390 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:38.103399 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:38.103419 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:38.103420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:38.103432 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:38.103493 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:38.103762 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.104560 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.105533 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.105837 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.105895 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.108327 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:51383: BlockingRecv error: recv got EOF from 127.10.130.193:51383 (error 108)
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":76,"threads_started":1}
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:38.112766 19608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:38.108516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:58259)
0924 07:14:38.108563 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:38.108574 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:38.108613 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:38.108742 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:38.108742 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:38.108871 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:38.108900 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.108901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.109726 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.109727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.110508 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.110509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.110821 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.110822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.110871 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.111201 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:38.111206 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:38.112707 (+ 1501us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 07:14:38.112881 19606 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 07:14:38.112963 19612 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:38.108596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:58259 (local address 127.10.130.254:34947)
0924 07:14:38.108632 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:38.108634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:38.108643 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:38.108660 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:38.108661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:38.108669 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:38.108728 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:38.108962 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.109636 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.110582 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.110761 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.110813 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.112788 (+ 1975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:58259: BlockingRecv error: recv got EOF from 127.10.130.193:58259 (error 108)
Metrics: {"server-negotiator.queue_time_us":13}
W20260924 07:14:38.175355 19610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:38.175555 19610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:38.175586 19610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:38.176805 19610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:38.176846 19610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:38.176878 19610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:38.178297 19610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.194.keytab
--principal=oryx/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:46701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=42331
--tserver_master_addrs=127.10.130.254:34947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:38.178526 19610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:38.178732 19610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:38.183308 19617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:38.183332 19620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:38.183318 19618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:38.183480 19610 server_base.cc:1034] running on GCE node
I20260924 07:14:38.183681 19610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:38.183874 19610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:38.185007 19610 hybrid_clock.cc:648] HybridClock initialized: now 1790234078184996 us; error 26 us; skew 500 ppm
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234078, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:38.187304 19610 init.cc:377] Logged in from keytab as oryx/127.10.130.194@KRBTEST.COM (short username oryx)
I20260924 07:14:38.187685 19610 webserver.cc:460] Webserver started at http://127.10.130.194:42331/ using document root <none> and password file <none>
I20260924 07:14:38.187855 19610 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:38.187909 19610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:38.189158 19610 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260924 07:14:38.189618 19627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:38.189769 19610 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:38.189845 19610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "7eba6b34aad1487399b2fef8f3b289cc"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "a1eddeddb23215a86ade28b890653913"
server_key_iv: "8dadc1701ca1af550fde785c9aecd4a7"
server_key_version: "encryptionkey@0"
I20260924 07:14:38.190157 19610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:38.202704 19610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:38.204617 19610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:38.204716 19610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:38.204877 19610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:38.205102 19610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:38.205135 19610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:38.205154 19610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:38.205168 19610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:38.210665 19610 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:46701
I20260924 07:14:38.210721 19740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:46701 every 8 connection(s)
I20260924 07:14:38.211035 19610 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
W20260924 07:14:38.217504 19612 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:38.211622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:43369 (local address 127.10.130.254:34947)
0924 07:14:38.211661 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:38.211663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:38.211869 (+ 206us) server_negotiation.cc:407] Connection header received
0924 07:14:38.212387 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:38.212388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:38.212403 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:38.212447 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:38.213310 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.213798 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.214728 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.214933 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.214988 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.217451 (+ 2463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:43369: BlockingRecv error: recv got EOF from 127.10.130.194:43369 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:14:38.217530 19743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:38.211614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:43369)
0924 07:14:38.211762 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:38.211795 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:38.212293 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:38.212526 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:38.212530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:38.213045 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:38.213220 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.213226 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.213873 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.213874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.214652 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.214654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.214997 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.214997 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.215118 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.215466 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:38.215473 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:38.217353 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
W20260924 07:14:38.217779 19741 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:34947 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.130.254:34947: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
I20260924 07:14:38.218622 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 19610
I20260924 07:14:38.218891 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:39799
--local_ip_for_outbound_sockets=127.10.130.195
--tserver_master_addrs=127.10.130.254:34947
--webserver_port=42375
--webserver_interface=127.10.130.195
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.130.195
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:38.221925 19743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:38.217969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:44573)
0924 07:14:38.218016 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:38.218027 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:38.218070 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:38.218170 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:38.218170 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:38.218269 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:38.218310 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.218311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.219050 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.219051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.219543 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.219545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.219933 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.219934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.220008 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.220325 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:38.220330 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:38.221875 (+ 1545us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 07:14:38.222023 19612 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:38.218020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:44573 (local address 127.10.130.254:34947)
0924 07:14:38.218063 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:38.218064 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:38.218070 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:14:38.218110 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:38.218111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:38.218119 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:38.218158 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:38.218354 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.218977 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.219613 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.219865 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.219930 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.221967 (+ 2037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:44573: BlockingRecv error: recv got EOF from 127.10.130.194:44573 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:38.226020 19743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:38.222131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:44715)
0924 07:14:38.222166 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:38.222174 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:38.222204 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:38.222334 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:38.222335 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:38.222432 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:38.222456 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.222457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.223269 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.223270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.223719 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.223720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.224144 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.224145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.224179 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.224514 (+ 335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:38.224518 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:38.225981 (+ 1463us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260924 07:14:38.226114 19741 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 07:14:38.226127 19612 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:38.222190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:44715 (local address 127.10.130.254:34947)
0924 07:14:38.222232 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:38.222234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:38.222240 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:14:38.222259 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:38.222260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:38.222269 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:38.222326 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:38.222516 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.223196 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.223791 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.224078 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.224140 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.226074 (+ 1934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:44715: BlockingRecv error: recv got EOF from 127.10.130.194:44715 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 07:14:38.297986 19744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:38.298173 19744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:38.298194 19744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:38.299268 19744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:38.299300 19744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:38.299324 19744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:38.300664 19744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/krb5kdc/oryx_127.10.130.195.keytab
--principal=oryx/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:39799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=42375
--tserver_master_addrs=127.10.130.254:34947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:38.300841 19744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:38.301009 19744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:38.305531 19749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:38.305531 19750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:38.305689 19744 server_base.cc:1034] running on GCE node
W20260924 07:14:38.305542 19752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:38.305912 19744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:38.306124 19744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:38.307266 19744 hybrid_clock.cc:648] HybridClock initialized: now 1790234078307253 us; error 28 us; skew 500 ppm
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234078, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:38.313860 19744 init.cc:377] Logged in from keytab as oryx/127.10.130.195@KRBTEST.COM (short username oryx)
I20260924 07:14:38.314263 19744 webserver.cc:460] Webserver started at http://127.10.130.195:42375/ using document root <none> and password file <none>
I20260924 07:14:38.314428 19744 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:38.314473 19744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:38.315639 19744 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:38.316146 19759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:38.316282 19744 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:38.316352 19744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "1990b10887744909a20d0831a8794094"
format_stamp: "Formatted at 2026-09-24 07:14:37 on dist-test-slave-7q7m"
server_key: "84eccdb24f344a655629cc538ef49489"
server_key_iv: "fba6033621c4c9d84fc1c89d3cf60875"
server_key_version: "encryptionkey@0"
I20260924 07:14:38.316665 19744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:38.338586 19744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:38.340487 19744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:38.340607 19744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:38.340829 19744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:38.341126 19744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:38.341158 19744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:38.341179 19744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:38.341202 19744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:38.346417 19744 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:39799
I20260924 07:14:38.346486 19872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:39799 every 8 connection(s)
I20260924 07:14:38.346808 19744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
W20260924 07:14:38.353794 19612 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:38.347448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:56729 (local address 127.10.130.254:34947)
0924 07:14:38.347510 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:38.347512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:38.347701 (+ 189us) server_negotiation.cc:407] Connection header received
0924 07:14:38.348257 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:38.348259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:38.348272 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:38.348319 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:38.349262 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.349751 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.350791 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.350996 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.351050 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.353740 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:56729: BlockingRecv error: recv got EOF from 127.10.130.195:56729 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 07:14:38.353838 19875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:38.347428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:56729)
0924 07:14:38.347582 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:38.347619 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:38.348147 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:38.348401 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:38.348406 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:38.348967 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:38.349155 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.349160 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.349830 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.349831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.350690 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.350692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.351070 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.351071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.351206 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.351676 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:38.351684 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:38.353644 (+ 1960us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260924 07:14:38.353837 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 19744
W20260924 07:14:38.354005 19873 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:34947 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.130.254:34947: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:38.358299 19875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:38.354215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:60721)
0924 07:14:38.354261 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:38.354271 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:38.354309 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:38.354492 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:38.354493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:38.354640 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:38.354683 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.354684 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.355291 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.355292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.355843 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.355844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.356219 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.356220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.356290 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.356643 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:38.356647 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:38.358241 (+ 1594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 07:14:38.358354 19612 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:38.354361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:60721 (local address 127.10.130.254:34947)
0924 07:14:38.354401 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:38.354402 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:38.354408 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:14:38.354427 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:38.354427 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:38.354436 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:38.354478 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:38.354751 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.355217 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.355913 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.356155 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.356204 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.358307 (+ 2103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:60721: BlockingRecv error: recv got EOF from 127.10.130.195:60721 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 07:14:38.360464 19609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:38.354352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53824 (local address 127.10.130.254:34947)
0924 07:14:38.354400 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:38.354401 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:38.354424 (+ 23us) server_negotiation.cc:407] Connection header received
0924 07:14:38.354490 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:38.354491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:38.354503 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:38.354547 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:38.355096 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.355558 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.356295 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.356471 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.356522 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.357538 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:38.357542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:38.357543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:38.357553 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:38.359178 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:38.359481 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:38.359484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:38.359485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:38.359576 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:38.359744 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:38.359745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:38.359745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:38.360046 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:38.360169 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:38.360282 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 07:14:38.360379 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:38 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:38.362402 19875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:38.358714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:36907)
0924 07:14:38.358754 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:38.358764 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:38.358802 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:38.358897 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:38.358898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:38.359038 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:38.359061 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.359062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.359698 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.359699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.360243 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:38.360244 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.360638 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.360639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:38.360692 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.361022 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:38.361026 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:38.362363 (+ 1337us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260924 07:14:38.362483 19612 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:38.358780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:36907 (local address 127.10.130.254:34947)
0924 07:14:38.358814 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:38.358816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:38.358822 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:14:38.358840 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:38.358841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:38.358849 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:38.358896 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:38.359123 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.359637 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.360306 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:38.360574 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:38.360629 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:38.362437 (+ 1808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:36907: BlockingRecv error: recv got EOF from 127.10.130.195:36907 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 07:14:38.362537 19873 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 24 07:14:39 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:39 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:39.124539 19877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:39.113229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:41833)
0924 07:14:39.113353 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:39.113362 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:39.113410 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:39.113526 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:39.113527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:39.113673 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:39.113742 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:39.113743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:39.114350 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:39.114350 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:39.114902 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:39.114904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:39.115191 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:39.115191 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:39.115233 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:39.122667 (+ 7434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:39.122673 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:39.124466 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
W20260924 07:14:39.124622 19878 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:39.113299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:41833 (local address 127.10.130.254:34947)
0924 07:14:39.113409 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:39.113410 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:39.113418 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:39.113452 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:39.113453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:39.113461 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:39.113515 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:39.113794 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:39.114285 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:39.114961 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:39.115144 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:39.115196 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:39.124559 (+ 9363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:41833: BlockingRecv error: recv got EOF from 127.10.130.193:41833 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":50,"threads_started":1}
Sep 24 07:14:39 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:39 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:39.230630 19879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:39.226430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:45227)
0924 07:14:39.226555 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:39.226564 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:39.226617 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:39.226729 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:39.226729 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:39.226878 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:39.226926 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:39.226928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:39.227517 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:39.227518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:39.228104 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:39.228107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:39.228535 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:39.228536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:39.228596 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:39.228982 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:39.228986 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:39.230567 (+ 1581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
W20260924 07:14:39.230787 19878 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:39.226505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:45227 (local address 127.10.130.254:34947)
0924 07:14:39.226559 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:39.226561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:39.226611 (+ 50us) server_negotiation.cc:407] Connection header received
0924 07:14:39.226670 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:39.226671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:39.226680 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:39.226717 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:39.226975 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:39.227452 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:39.228181 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:39.228462 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:39.228526 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:39.230642 (+ 2116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:45227: BlockingRecv error: recv got EOF from 127.10.130.194:45227 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 24 07:14:39 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:39 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:39.367101 19880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:39.362842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:49125)
0924 07:14:39.362959 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:39.362970 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:39.363034 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:39.363150 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:39.363151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:39.363312 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:39.363361 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:39.363362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:39.363944 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:39.363944 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:39.364527 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:39.364529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:39.364819 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:39.364820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:39.364860 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:39.365371 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:39.365378 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:39.367013 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":53,"threads_started":1}
W20260924 07:14:39.367151 19878 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:39.362903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:49125 (local address 127.10.130.254:34947)
0924 07:14:39.362969 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:39.362971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:39.363024 (+ 53us) server_negotiation.cc:407] Connection header received
0924 07:14:39.363081 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:39.363082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:39.363093 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:39.363136 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:39.363415 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:39.363886 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:39.364588 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:39.364768 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:39.364813 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:39.367100 (+ 2287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:49125: BlockingRecv error: recv got EOF from 127.10.130.195:49125 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 24 07:14:40 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:40 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:40.129349 19881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:40.124983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:59909)
0924 07:14:40.125136 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:40.125147 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:40.125207 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:40.125322 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:40.125323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:40.125479 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:40.125532 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:40.125533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:40.126218 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:40.126219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:40.126748 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:40.126749 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:40.127096 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:40.127097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:40.127134 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:40.127552 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:40.127556 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:40.129284 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
W20260924 07:14:40.129429 19882 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:40.125076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:59909 (local address 127.10.130.254:34947)
0924 07:14:40.125179 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:40.125180 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:40.125192 (+ 12us) server_negotiation.cc:407] Connection header received
0924 07:14:40.125251 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:40.125252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:40.125262 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:40.125316 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:40.125621 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:40.126118 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:40.126833 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:40.127041 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:40.127095 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:40.129367 (+ 2272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:59909: BlockingRecv error: recv got EOF from 127.10.130.193:59909 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":45,"threads_started":1}
Sep 24 07:14:40 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:40 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:40.234869 19883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:40.231038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:56241)
0924 07:14:40.231145 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:40.231156 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:40.231209 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:40.231319 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:40.231320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:40.231467 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:40.231513 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:40.231515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:40.232136 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:40.232137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:40.232656 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:40.232658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:40.232932 (+ 274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:40.232933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:40.232970 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:40.233336 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:40.233341 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:40.234812 (+ 1471us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":62,"threads_started":1}
W20260924 07:14:40.235037 19882 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:40.231121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:56241 (local address 127.10.130.254:34947)
0924 07:14:40.231171 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:40.231173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:40.231203 (+ 30us) server_negotiation.cc:407] Connection header received
0924 07:14:40.231251 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:40.231252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:40.231262 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:40.231308 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:40.231560 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:40.232086 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:40.232707 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:40.232883 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:40.232927 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:40.234898 (+ 1971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:56241: BlockingRecv error: recv got EOF from 127.10.130.194:56241 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 24 07:14:40 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:40 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:40.371845 19884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:40.367595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:47033)
0924 07:14:40.367727 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:40.367740 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:40.367813 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:40.367939 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:40.367940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:40.368136 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:40.368184 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:40.368187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:40.368800 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:40.368801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:40.369321 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:40.369323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:40.369641 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:40.369642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:40.369693 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:40.370123 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:40.370128 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:40.371768 (+ 1640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":56,"threads_started":1}
W20260924 07:14:40.371932 19882 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:40.367713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:47033 (local address 127.10.130.254:34947)
0924 07:14:40.367755 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:40.367758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:40.367792 (+ 34us) server_negotiation.cc:407] Connection header received
0924 07:14:40.367862 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:40.367863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:40.367875 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:40.367928 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:40.368252 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:40.368731 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:40.369396 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:40.369582 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:40.369637 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:40.371869 (+ 2232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:47033: BlockingRecv error: recv got EOF from 127.10.130.195:47033 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 24 07:14:41 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:41 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:41.133908 19885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:41.129757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:48765)
0924 07:14:41.129880 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:41.129890 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:41.129935 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:41.130053 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:41.130054 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:41.130202 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:41.130253 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:41.130254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:41.130864 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:41.130865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:41.131377 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:41.131379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:41.131655 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:41.131656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:41.131692 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:41.132168 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:41.132172 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:41.133848 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":57,"threads_started":1}
W20260924 07:14:41.134011 19886 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:41.129827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:48765 (local address 127.10.130.254:34947)
0924 07:14:41.129938 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:41.129940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:41.129949 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:41.129972 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:41.129972 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:41.129983 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:41.130043 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:41.130319 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:41.130810 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:41.131430 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:41.131609 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:41.131650 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:41.133942 (+ 2292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:48765: BlockingRecv error: recv got EOF from 127.10.130.193:48765 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
Sep 24 07:14:41 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:41 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:41.239553 19887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:41.235331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:41347)
0924 07:14:41.235477 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:41.235487 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:41.235544 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:41.235682 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:41.235683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:41.235880 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:41.235937 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:41.235940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:41.236590 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:41.236590 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:41.237110 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:41.237112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:41.237398 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:41.237398 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:41.237439 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:41.237845 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:41.237850 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:41.239495 (+ 1645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
W20260924 07:14:41.239650 19886 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:41.235404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:41347 (local address 127.10.130.254:34947)
0924 07:14:41.235455 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:41.235457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:41.235536 (+ 79us) server_negotiation.cc:407] Connection header received
0924 07:14:41.235600 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:41.235601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:41.235613 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:41.235667 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:41.236027 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:41.236526 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:41.237164 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:41.237350 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:41.237396 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:41.239582 (+ 2186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:41347: BlockingRecv error: recv got EOF from 127.10.130.194:41347 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 24 07:14:41 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:41 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:41.376632 19888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:41.372327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:50759)
0924 07:14:41.372450 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:41.372462 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:41.372534 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:41.372662 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:41.372663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:41.372828 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:41.372874 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:41.372876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:41.373513 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:41.373515 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:41.374137 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:41.374140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:41.374461 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:41.374462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:41.374518 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:41.374925 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:41.374929 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:41.376574 (+ 1645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
W20260924 07:14:41.376721 19886 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:41.372398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:50759 (local address 127.10.130.254:34947)
0924 07:14:41.372462 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:41.372464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:41.372514 (+ 50us) server_negotiation.cc:407] Connection header received
0924 07:14:41.372585 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:41.372586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:41.372600 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:41.372648 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:41.372948 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:41.373443 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:41.374207 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:41.374395 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:41.374452 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:41.376661 (+ 2209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:50759: BlockingRecv error: recv got EOF from 127.10.130.195:50759 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 24 07:14:42 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:42 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:42.138470 19889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:42.134379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:54323)
0924 07:14:42.134526 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:42.134536 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:42.134593 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:42.134716 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:42.134716 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:42.134878 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:42.134930 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:42.134932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:42.135529 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:42.135530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:42.136121 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:42.136123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:42.136417 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:42.136418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:42.136455 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:42.136848 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:42.136852 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:42.138413 (+ 1561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
W20260924 07:14:42.138576 19890 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:42.134470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:54323 (local address 127.10.130.254:34947)
0924 07:14:42.134572 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:42.134574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:42.134582 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:42.134652 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:42.134653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:42.134663 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:42.134709 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:42.134986 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:42.135473 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:42.136178 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:42.136367 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:42.136413 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:42.138508 (+ 2095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:54323: BlockingRecv error: recv got EOF from 127.10.130.193:54323 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":56,"threads_started":1}
Sep 24 07:14:42 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:42 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:42.244329 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:42.239965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:32891)
0924 07:14:42.240127 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:42.240137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:42.240194 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:42.240325 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:42.240326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:42.240506 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:42.240564 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:42.240567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:42.241391 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:42.241392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:42.241937 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:42.241939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:42.242330 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:42.242331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:42.242369 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:42.242736 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:42.242739 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:42.244261 (+ 1522us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":54,"threads_started":1}
W20260924 07:14:42.244386 19890 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:42.240029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:32891 (local address 127.10.130.254:34947)
0924 07:14:42.240078 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:42.240080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:42.240191 (+ 111us) server_negotiation.cc:407] Connection header received
0924 07:14:42.240253 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:42.240254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:42.240276 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:42.240321 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:42.240633 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:42.241292 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:42.242007 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:42.242258 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:42.242326 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:42.244340 (+ 2014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:32891: BlockingRecv error: recv got EOF from 127.10.130.194:32891 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 24 07:14:42 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:42 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:42.381260 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:42.377071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:45279)
0924 07:14:42.377214 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:42.377225 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:42.377286 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:42.377432 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:42.377433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:42.377630 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:42.377689 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:42.377692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:42.378366 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:42.378367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:42.378894 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:42.378895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:42.379177 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:42.379178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:42.379213 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:42.379612 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:42.379616 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:42.381201 (+ 1585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
W20260924 07:14:42.381378 19890 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:42.377164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:45279 (local address 127.10.130.254:34947)
0924 07:14:42.377215 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:42.377217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:42.377276 (+ 59us) server_negotiation.cc:407] Connection header received
0924 07:14:42.377354 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:42.377355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:42.377369 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:42.377417 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:42.377806 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:42.378301 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:42.378951 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:42.379127 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:42.379176 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:42.381313 (+ 2137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:45279: BlockingRecv error: recv got EOF from 127.10.130.195:45279 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 24 07:14:43 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:43 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:43.143023 19893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:43.138953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:55843)
0924 07:14:43.139080 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:43.139091 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:43.139147 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:43.139287 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:43.139288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:43.139439 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:43.139487 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:43.139488 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:43.140168 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:43.140168 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:43.140687 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:43.140688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:43.141002 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:43.141002 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:43.141038 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:43.141429 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:43.141433 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:43.142966 (+ 1533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
W20260924 07:14:43.143155 19894 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:43.139053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:55843 (local address 127.10.130.254:34947)
0924 07:14:43.139172 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:43.139174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:43.139182 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:43.139203 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:43.139204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:43.139214 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:43.139281 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:43.139557 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:43.140109 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:43.140745 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:43.140951 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:43.141003 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:43.143075 (+ 2072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:55843: BlockingRecv error: recv got EOF from 127.10.130.193:55843 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
Sep 24 07:14:43 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:43 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:43.249018 19895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:43.244801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:36727)
0924 07:14:43.244959 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:43.244971 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:43.245034 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:43.245161 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:43.245161 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:43.245328 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:43.245379 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:43.245380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:43.245972 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:43.245973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:43.246520 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:43.246522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:43.246812 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:43.246813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:43.246854 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:43.247274 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:43.247278 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:43.248958 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":65,"threads_started":1}
W20260924 07:14:43.249115 19894 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:43.244909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:36727 (local address 127.10.130.254:34947)
0924 07:14:43.244975 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:43.244977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:43.245017 (+ 40us) server_negotiation.cc:407] Connection header received
0924 07:14:43.245079 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:43.245080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:43.245094 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:43.245141 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:43.245433 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:43.245913 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:43.246580 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:43.246756 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:43.246798 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:43.249056 (+ 2258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:36727: BlockingRecv error: recv got EOF from 127.10.130.194:36727 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 24 07:14:43 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:43 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:43.386009 19896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:43.381657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:43545)
0924 07:14:43.381774 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:43.381784 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:43.381830 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:43.381962 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:43.381963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:43.382115 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:43.382163 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:43.382165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:43.382834 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:43.382835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:43.383351 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:43.383353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:43.383694 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:43.383695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:43.383731 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:43.384219 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:43.384224 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:43.385949 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":66,"threads_started":1}
W20260924 07:14:43.386106 19894 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:43.381749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:43545 (local address 127.10.130.254:34947)
0924 07:14:43.381839 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:43.381841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:43.381875 (+ 34us) server_negotiation.cc:407] Connection header received
0924 07:14:43.381899 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:43.381899 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:43.381913 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:43.381959 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:43.382224 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:43.382772 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:43.383435 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:43.383633 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:43.383692 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:43.386040 (+ 2348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:43545: BlockingRecv error: recv got EOF from 127.10.130.195:43545 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 24 07:14:44 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:44 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:44.150822 19897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:44.143484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:43015)
0924 07:14:44.143605 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:44.143615 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:44.143671 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:44.143782 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:44.143783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:44.143935 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:44.144021 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:44.144023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:44.144745 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:44.144745 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:44.145394 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:44.145396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:44.145739 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:44.145739 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:44.145776 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:44.149087 (+ 3311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:44.149091 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:44.150765 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
W20260924 07:14:44.150950 19898 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:44.143574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:43015 (local address 127.10.130.254:34947)
0924 07:14:44.143666 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:44.143669 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:44.143678 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:44.143707 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:44.143708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:44.143718 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:44.143774 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:44.144114 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:44.144637 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:44.145450 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:44.145647 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:44.145731 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:44.150869 (+ 5138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:43015: BlockingRecv error: recv got EOF from 127.10.130.193:43015 (error 108)
Metrics: {"server-negotiator.queue_time_us":73,"thread_start_us":43,"threads_started":1}
Sep 24 07:14:44 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:44 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:44.253958 19899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:44.249493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:42327)
0924 07:14:44.249656 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:44.249668 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:44.249731 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:44.249878 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:44.249879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:44.250052 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:44.250105 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:44.250107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:44.250730 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:44.250731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:44.251393 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:44.251397 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:44.251766 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:44.251768 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:44.251853 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:44.252278 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:44.252283 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:44.253899 (+ 1616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":82,"threads_started":1}
W20260924 07:14:44.254047 19898 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:44.249544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:42327 (local address 127.10.130.254:34947)
0924 07:14:44.249594 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:44.249596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:44.249720 (+ 124us) server_negotiation.cc:407] Connection header received
0924 07:14:44.249806 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:44.249806 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:44.249816 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:44.249859 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:44.250158 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:44.250648 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:44.251477 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:44.251683 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:44.251737 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:44.253987 (+ 2250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:42327: BlockingRecv error: recv got EOF from 127.10.130.194:42327 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 24 07:14:44 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:44 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:44.390789 19900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:44.386441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:43253)
0924 07:14:44.386592 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:44.386603 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:44.386669 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:44.386788 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:44.386788 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:44.386961 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:44.387004 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:44.387006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:44.387650 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:44.387651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:44.388241 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:44.388243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:44.388570 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:44.388571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:44.388624 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:44.389021 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:44.389025 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:44.390715 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
W20260924 07:14:44.390866 19898 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:44.386529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:43253 (local address 127.10.130.254:34947)
0924 07:14:44.386581 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:44.386584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:44.386650 (+ 66us) server_negotiation.cc:407] Connection header received
0924 07:14:44.386717 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:44.386718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:44.386731 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:44.386774 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:44.387075 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:44.387579 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:44.388311 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:44.388512 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:44.388566 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:44.390814 (+ 2248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:43253: BlockingRecv error: recv got EOF from 127.10.130.195:43253 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 24 07:14:45 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:45 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:45.155334 19901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:45.151231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:58643)
0924 07:14:45.151356 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:45.151366 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:45.151413 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:45.151530 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:45.151530 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:45.151682 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:45.151731 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:45.151732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:45.152346 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:45.152347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:45.152886 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:45.152888 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:45.153176 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:45.153176 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:45.153211 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:45.153645 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:45.153650 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:45.155269 (+ 1619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
W20260924 07:14:45.155427 19902 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:45.151288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:58643 (local address 127.10.130.254:34947)
0924 07:14:45.151417 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:45.151418 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:45.151428 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:14:45.151451 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:45.151451 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:45.151462 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:45.151521 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:45.151779 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:45.152282 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:45.152943 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:45.153127 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:45.153173 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:45.155355 (+ 2182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:58643: BlockingRecv error: recv got EOF from 127.10.130.193:58643 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
Sep 24 07:14:45 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:45 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:45.258374 19903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:45.254370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:38103)
0924 07:14:45.254525 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:45.254535 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:45.254584 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:45.254708 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:45.254709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:45.254857 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:45.254905 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:45.254906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:45.255495 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:45.255496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:45.256032 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:45.256033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:45.256326 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:45.256327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:45.256361 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:45.256753 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:45.256757 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:45.258309 (+ 1552us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
W20260924 07:14:45.258457 19902 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:45.254457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:38103 (local address 127.10.130.254:34947)
0924 07:14:45.254539 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:45.254546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:45.254573 (+ 27us) server_negotiation.cc:407] Connection header received
0924 07:14:45.254634 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:45.254634 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:45.254645 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:45.254697 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:45.254953 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:45.255443 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:45.256078 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:45.256255 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:45.256317 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:45.258397 (+ 2080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:38103: BlockingRecv error: recv got EOF from 127.10.130.194:38103 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 24 07:14:45 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:45 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:45.395732 19904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:45.391255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:40129)
0924 07:14:45.391393 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:45.391403 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:45.391459 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:45.391620 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:45.391621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:45.391813 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:45.391874 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:45.391877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:45.392608 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:45.392609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:45.393147 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:45.393148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:45.393482 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:45.393483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:45.393524 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:45.393974 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:45.393978 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:45.395652 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
W20260924 07:14:45.395819 19902 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:45.391300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:40129 (local address 127.10.130.254:34947)
0924 07:14:45.391369 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:45.391371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:45.391469 (+ 98us) server_negotiation.cc:407] Connection header received
0924 07:14:45.391526 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:45.391527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:45.391544 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:45.391612 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:45.391948 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:45.392545 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:45.393212 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:45.393420 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:45.393477 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:45.395752 (+ 2275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:40129: BlockingRecv error: recv got EOF from 127.10.130.195:40129 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 24 07:14:46 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:46 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:46.160084 19905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:46.155785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:52379)
0924 07:14:46.155918 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:46.155928 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:46.156012 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:46.156147 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:46.156149 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:46.156291 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:46.156340 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:46.156342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:46.157007 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:46.157008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:46.157548 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:46.157550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:46.157894 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:46.157896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:46.157948 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:46.158340 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:46.158344 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:46.160025 (+ 1681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
W20260924 07:14:46.160166 19906 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:46.155901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:52379 (local address 127.10.130.254:34947)
0924 07:14:46.156028 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:46.156030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:46.156039 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:46.156066 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:46.156066 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:46.156076 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:46.156137 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:46.156394 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:46.156926 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:46.157629 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:46.157833 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:46.157886 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:46.160102 (+ 2216us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:52379: BlockingRecv error: recv got EOF from 127.10.130.193:52379 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
Sep 24 07:14:46 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:46 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:46.263199 19907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:46.258799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:57017)
0924 07:14:46.258935 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:46.258945 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:46.259000 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:46.259115 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:46.259115 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:46.259263 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:46.259310 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:46.259312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:46.260135 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:46.260136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:46.260672 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:46.260674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:46.260952 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:46.260952 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:46.260989 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:46.261368 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:46.261372 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:46.263123 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
W20260924 07:14:46.263264 19906 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:46.258846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:57017 (local address 127.10.130.254:34947)
0924 07:14:46.258913 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:46.258915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:46.258996 (+ 81us) server_negotiation.cc:407] Connection header received
0924 07:14:46.259053 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:46.259054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:46.259064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:46.259103 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:46.259386 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:46.260066 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:46.260728 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:46.260906 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:46.260948 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:46.263213 (+ 2265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:57017: BlockingRecv error: recv got EOF from 127.10.130.194:57017 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 24 07:14:46 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:46 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:46.400575 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:46.396189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:58867)
0924 07:14:46.396357 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:46.396369 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:46.396448 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:46.396600 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:46.396601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:46.396812 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:46.396864 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:46.396866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:46.397549 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:46.397550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:46.398085 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:46.398087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:46.398426 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:46.398426 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:46.398480 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:46.398876 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:46.398880 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:46.400517 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":71,"threads_started":1}
W20260924 07:14:46.400682 19906 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:46.396258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:58867 (local address 127.10.130.254:34947)
0924 07:14:46.396354 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:46.396357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:46.396434 (+ 77us) server_negotiation.cc:407] Connection header received
0924 07:14:46.396515 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:46.396516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:46.396533 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:46.396588 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:46.396941 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:46.397472 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:46.398165 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:46.398360 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:46.398423 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:46.400608 (+ 2185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:58867: BlockingRecv error: recv got EOF from 127.10.130.195:58867 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 24 07:14:47 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:47 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:47.164829 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:47.160516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:58889)
0924 07:14:47.160647 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:47.160657 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:47.160707 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:47.160859 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:47.160859 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:47.161017 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:47.161065 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:47.161067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:47.161696 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:47.161697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:47.162228 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:47.162229 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:47.162579 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:47.162580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:47.162615 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:47.163059 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:47.163062 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:47.164771 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
W20260924 07:14:47.164978 19910 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:47.160576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:58889 (local address 127.10.130.254:34947)
0924 07:14:47.160717 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:47.160719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:47.160746 (+ 27us) server_negotiation.cc:407] Connection header received
0924 07:14:47.160768 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:47.160768 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:47.160790 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:47.160850 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:47.161124 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:47.161624 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:47.162315 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:47.162517 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:47.162579 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:47.164879 (+ 2300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:58889: BlockingRecv error: recv got EOF from 127.10.130.193:58889 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
Sep 24 07:14:47 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:47 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:47.267855 19911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:47.263623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:43667)
0924 07:14:47.263743 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:47.263754 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:47.263822 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:47.263940 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:47.263941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:47.264147 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:47.264196 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:47.264198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:47.264817 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:47.264818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:47.265335 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:47.265337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:47.265660 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:47.265661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:47.265716 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:47.266170 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:47.266175 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:47.267771 (+ 1596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
W20260924 07:14:47.267925 19910 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:47.263692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:43667 (local address 127.10.130.254:34947)
0924 07:14:47.263745 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:47.263748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:47.263809 (+ 61us) server_negotiation.cc:407] Connection header received
0924 07:14:47.263870 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:47.263871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:47.263885 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:47.263930 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:47.264265 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:47.264747 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:47.265408 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:47.265602 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:47.265678 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:47.267872 (+ 2194us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:43667: BlockingRecv error: recv got EOF from 127.10.130.194:43667 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 24 07:14:47 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:47 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:47.405181 19912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:47.400946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:57041)
0924 07:14:47.401090 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:47.401101 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:47.401162 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:47.401277 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:47.401278 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:47.401456 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:47.401501 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:47.401502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:47.402135 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:47.402140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:47.402680 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:47.402682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:47.402953 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:47.402953 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:47.402991 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:47.403390 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:47.403394 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:47.405117 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":63,"threads_started":1}
W20260924 07:14:47.405257 19910 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:47.401012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:57041 (local address 127.10.130.254:34947)
0924 07:14:47.401073 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:47.401074 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:47.401153 (+ 79us) server_negotiation.cc:407] Connection header received
0924 07:14:47.401201 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:47.401202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:47.401218 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:47.401265 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:47.401554 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:47.402056 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:47.402734 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:47.402909 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:47.402950 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:47.405199 (+ 2249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:57041: BlockingRecv error: recv got EOF from 127.10.130.195:57041 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 24 07:14:48 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:48 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:48.169612 19913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:48.165255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:52665)
0924 07:14:48.165390 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:48.165400 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:48.165448 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:48.165572 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:48.165573 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:48.165734 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:48.165782 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:48.165784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:48.166418 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:48.166419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:48.166954 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:48.166957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:48.167391 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:48.167392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:48.167455 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:48.167906 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:48.167911 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:48.169556 (+ 1645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
W20260924 07:14:48.169739 19914 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:48.165340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:52665 (local address 127.10.130.254:34947)
0924 07:14:48.165439 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:48.165441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:48.165450 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:48.165493 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:48.165493 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:48.165504 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:48.165559 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:48.165853 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:48.166359 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:48.167030 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:48.167318 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:48.167389 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:48.169661 (+ 2272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:52665: BlockingRecv error: recv got EOF from 127.10.130.193:52665 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":48,"threads_started":1}
Sep 24 07:14:48 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:48 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:48.272655 19915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:48.268298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:50397)
0924 07:14:48.268428 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:48.268440 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:48.268515 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:48.268634 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:48.268635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:48.268800 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:48.268844 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:48.268846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:48.269480 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:48.269481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:48.270005 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:48.270007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:48.270317 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:48.270318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:48.270372 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:48.270790 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:48.270795 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:48.272598 (+ 1803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":78,"threads_started":1}
W20260924 07:14:48.272802 19914 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:48.268378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:50397 (local address 127.10.130.254:34947)
0924 07:14:48.268432 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:48.268434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:48.268484 (+ 50us) server_negotiation.cc:407] Connection header received
0924 07:14:48.268565 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:48.268566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:48.268579 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:48.268624 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:48.268909 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:48.269411 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:48.270076 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:48.270264 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:48.270315 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:48.272739 (+ 2424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:50397: BlockingRecv error: recv got EOF from 127.10.130.194:50397 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 24 07:14:48 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:48 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:48.409751 19916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:48.405597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:33825)
0924 07:14:48.405717 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:48.405727 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:48.405778 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:48.405890 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:48.405891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:48.406067 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:48.406112 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:48.406114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:48.406739 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:48.406740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:48.407259 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:48.407261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:48.407587 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:48.407588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:48.407641 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:48.408085 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:48.408089 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:48.409676 (+ 1587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":55,"threads_started":1}
W20260924 07:14:48.409817 19914 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:48.405727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:33825 (local address 127.10.130.254:34947)
0924 07:14:48.405786 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:48.405788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:48.405797 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:48.405830 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:48.405831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:48.405844 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:48.405885 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:48.406178 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:48.406670 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:48.407328 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:48.407519 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:48.407583 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:48.409764 (+ 2181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:33825: BlockingRecv error: recv got EOF from 127.10.130.195:33825 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 24 07:14:49 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:49 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:49.179841 19917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:49.170024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:44875)
0924 07:14:49.170166 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:49.170177 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:49.170236 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:49.170350 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:49.170351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:49.170504 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:49.170549 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:49.170550 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:49.171138 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:49.171139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:49.171645 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:49.171646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:49.171918 (+ 272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:49.171919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:49.171953 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:49.178060 (+ 6107us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:49.178065 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:49.179774 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
W20260924 07:14:49.179962 19918 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:49.170074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:44875 (local address 127.10.130.254:34947)
0924 07:14:49.170177 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:49.170179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:49.170221 (+ 42us) server_negotiation.cc:407] Connection header received
0924 07:14:49.170275 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:49.170276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:49.170287 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:49.170337 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:49.170602 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:49.171089 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:49.171697 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:49.171876 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:49.171917 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:49.179887 (+ 7970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:44875: BlockingRecv error: recv got EOF from 127.10.130.193:44875 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":57,"threads_started":1}
Sep 24 07:14:49 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:49 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:49.277400 19919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:49.273046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:35921)
0924 07:14:49.273175 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:49.273187 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:49.273263 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:49.273388 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:49.273389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:49.273555 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:49.273597 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:49.273599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:49.274217 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:49.274218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:49.274751 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:49.274753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:49.275073 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:49.275074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:49.275128 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:49.275532 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:49.275536 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:49.277319 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
W20260924 07:14:49.277468 19918 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:49.273139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:35921 (local address 127.10.130.254:34947)
0924 07:14:49.273192 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:49.273193 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:49.273251 (+ 58us) server_negotiation.cc:407] Connection header received
0924 07:14:49.273325 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:49.273326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:49.273337 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:49.273376 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:49.273664 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:49.274147 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:49.274825 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:49.275015 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:49.275069 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:49.277407 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:35921: BlockingRecv error: recv got EOF from 127.10.130.194:35921 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 24 07:14:49 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:49 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:49.414633 19920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:49.410213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:55941)
0924 07:14:49.410359 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:49.410368 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:49.410427 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:49.410551 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:49.410552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:49.410712 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:49.410758 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:49.410760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:49.411390 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:49.411391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:49.412028 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:49.412031 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:49.412339 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:49.412340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:49.412386 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:49.412880 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:49.412885 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:49.414560 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":63,"threads_started":1}
W20260924 07:14:49.414705 19918 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:49.410263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:55941 (local address 127.10.130.254:34947)
0924 07:14:49.410313 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:49.410315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:49.410422 (+ 107us) server_negotiation.cc:407] Connection header received
0924 07:14:49.410479 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:49.410479 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:49.410492 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:49.410536 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:49.410806 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:49.411291 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:49.412103 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:49.412275 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:49.412320 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:49.414643 (+ 2323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:55941: BlockingRecv error: recv got EOF from 127.10.130.195:55941 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 24 07:14:50 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:50 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:50.184314 19921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:50.180286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:36301)
0924 07:14:50.180410 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:50.180422 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:50.180486 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:50.180617 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:50.180617 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:50.180776 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:50.180821 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:50.180823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:50.181432 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:50.181433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:50.181954 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:50.181955 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:50.182235 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:50.182235 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:50.182271 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:50.182662 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:50.182665 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:50.184257 (+ 1592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
W20260924 07:14:50.184409 19922 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:50.180381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:36301 (local address 127.10.130.254:34947)
0924 07:14:50.180508 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:50.180510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:50.180518 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:50.180538 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:50.180539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:50.180549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:50.180607 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:50.180888 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:50.181376 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:50.182012 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:50.182191 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:50.182232 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:50.184343 (+ 2111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:36301: BlockingRecv error: recv got EOF from 127.10.130.193:36301 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
Sep 24 07:14:50 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:50 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:50.282208 19923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:50.277829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:37393)
0924 07:14:50.277958 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:50.277971 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:50.278051 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:50.278199 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:50.278200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:50.278382 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:50.278429 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:50.278431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:50.279080 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:50.279081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:50.279628 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:50.279630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:50.279965 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:50.279966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:50.280049 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:50.280448 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:50.280452 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:50.282114 (+ 1662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
W20260924 07:14:50.282274 19922 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:50.277910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:37393 (local address 127.10.130.254:34947)
0924 07:14:50.277968 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:50.277971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:50.278035 (+ 64us) server_negotiation.cc:407] Connection header received
0924 07:14:50.278121 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:50.278122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:50.278136 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:50.278189 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:50.278504 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:50.279000 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:50.279709 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:50.279901 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:50.279963 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:50.282219 (+ 2256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:37393: BlockingRecv error: recv got EOF from 127.10.130.194:37393 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 24 07:14:50 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:50 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:50.419317 19924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:50.415062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:59191)
0924 07:14:50.415179 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:50.415192 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:50.415259 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:50.415388 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:50.415388 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:50.415554 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:50.415599 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:50.415600 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:50.416304 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:50.416305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:50.416827 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:50.416829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:50.417133 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:50.417133 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:50.417185 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:50.417588 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:50.417592 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:50.419242 (+ 1650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
W20260924 07:14:50.419399 19922 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:50.415133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:59191 (local address 127.10.130.254:34947)
0924 07:14:50.415207 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:50.415210 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:50.415241 (+ 31us) server_negotiation.cc:407] Connection header received
0924 07:14:50.415313 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:50.415314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:50.415328 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:50.415371 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:50.415665 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:50.416226 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:50.416896 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:50.417078 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:50.417131 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:50.419343 (+ 2212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:59191: BlockingRecv error: recv got EOF from 127.10.130.195:59191 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 24 07:14:51 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:51 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:51.189018 19925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:51.184721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:49535)
0924 07:14:51.184857 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:51.184867 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:51.184944 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:51.185073 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:51.185074 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:51.185239 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:51.185287 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:51.185288 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:51.185956 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:51.185957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:51.186503 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:51.186505 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:51.186856 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:51.186857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:51.186917 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:51.187303 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:51.187307 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:51.188960 (+ 1653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
W20260924 07:14:51.189133 19926 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:51.184780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:49535 (local address 127.10.130.254:34947)
0924 07:14:51.184876 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:51.184878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:51.184937 (+ 59us) server_negotiation.cc:407] Connection header received
0924 07:14:51.185000 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:51.185000 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:51.185011 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:51.185060 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:51.185334 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:51.185895 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:51.186595 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:51.186806 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:51.186852 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:51.189056 (+ 2204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:49535: BlockingRecv error: recv got EOF from 127.10.130.193:49535 (error 108)
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":44,"threads_started":1}
Sep 24 07:14:51 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:51 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:51.287309 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:51.282734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:50061)
0924 07:14:51.282872 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:51.282882 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:51.282940 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:51.283074 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:51.283075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:51.283242 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:51.283287 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:51.283289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:51.283932 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:51.283934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:51.284487 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:51.284489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:51.284874 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:51.284875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:51.284930 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:51.285377 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:51.285382 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:51.287250 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
W20260924 07:14:51.287389 19926 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:51.282874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:50061 (local address 127.10.130.254:34947)
0924 07:14:51.282943 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:51.282946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:51.282955 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:51.283004 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:51.283006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:51.283016 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:51.283063 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:51.283363 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:51.283860 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:51.284574 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:51.284802 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:51.284854 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:51.287328 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:50061: BlockingRecv error: recv got EOF from 127.10.130.194:50061 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 24 07:14:51 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:51 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:51.424098 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:51.419800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:55981)
0924 07:14:51.419955 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:51.419967 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:51.420057 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:51.420189 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:51.420190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:51.420369 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:51.420416 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:51.420418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:51.421060 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:51.421062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:51.421584 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:51.421586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:51.421915 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:51.421916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:51.421970 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:51.422364 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:51.422368 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:51.424020 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
W20260924 07:14:51.424156 19926 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:51.419898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:55981 (local address 127.10.130.254:34947)
0924 07:14:51.419959 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:51.419961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:51.420038 (+ 77us) server_negotiation.cc:407] Connection header received
0924 07:14:51.420115 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:51.420116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:51.420134 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:51.420182 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:51.420490 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:51.420987 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:51.421665 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:51.421857 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:51.421913 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:51.424104 (+ 2191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:55981: BlockingRecv error: recv got EOF from 127.10.130.195:55981 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 24 07:14:52 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:52 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:52.193933 19929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:52.189445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:43527)
0924 07:14:52.189613 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:52.189625 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:52.189687 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:52.189805 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:52.189806 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:52.189978 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:52.190030 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:52.190031 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:52.190660 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:52.190661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:52.191198 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:52.191199 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:52.191520 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:52.191521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:52.191565 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:52.191970 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:52.191998 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:52.193858 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":86,"threads_started":1}
W20260924 07:14:52.194006 19930 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:52.189492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:43527 (local address 127.10.130.254:34947)
0924 07:14:52.189618 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:52.189620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:52.189682 (+ 62us) server_negotiation.cc:407] Connection header received
0924 07:14:52.189728 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:52.189729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:52.189740 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:52.189794 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:52.190083 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:52.190578 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:52.191266 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:52.191456 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:52.191504 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:52.193939 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:43527: BlockingRecv error: recv got EOF from 127.10.130.193:43527 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":55,"threads_started":1}
Sep 24 07:14:52 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:52 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:52.292160 19931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:52.287758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:47717)
0924 07:14:52.287895 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:52.287905 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:52.288007 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:52.288185 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:52.288186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:52.288345 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:52.288392 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:52.288394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:52.288994 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:52.288995 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:52.289696 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:52.289699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:52.290023 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:52.290023 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:52.290070 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:52.290447 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:52.290451 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:52.292091 (+ 1640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":57,"threads_started":1}
W20260924 07:14:52.292237 19930 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:52.287805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:47717 (local address 127.10.130.254:34947)
0924 07:14:52.287858 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:52.287860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:52.287961 (+ 101us) server_negotiation.cc:407] Connection header received
0924 07:14:52.288101 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:52.288103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:52.288116 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:52.288172 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:52.288445 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:52.288931 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:52.289777 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:52.289974 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:52.290020 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:52.292177 (+ 2157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:47717: BlockingRecv error: recv got EOF from 127.10.130.194:47717 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 24 07:14:52 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:52 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.195@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:52.428893 19932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:52.424505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.195:55127)
0924 07:14:52.424637 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:52.424665 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:52.424724 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:52.424836 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:52.424837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:52.425046 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:52.425104 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:52.425106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:52.425824 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:52.425826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:52.426393 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:52.426395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:52.426687 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:52.426688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:52.426726 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:52.427194 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:52.427200 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:52.428827 (+ 1627us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
W20260924 07:14:52.428979 19930 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:52.424581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:55127 (local address 127.10.130.254:34947)
0924 07:14:52.424646 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:52.424648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:52.424706 (+ 58us) server_negotiation.cc:407] Connection header received
0924 07:14:52.424772 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:52.424773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:52.424786 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:52.424829 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:52.425187 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:52.425741 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:52.426460 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:52.426635 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:52.426682 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:52.428906 (+ 2224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.195:55127: BlockingRecv error: recv got EOF from 127.10.130.195:55127 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 24 07:14:53 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:53 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.193@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:53.198760 19933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:53.194437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.193:37953)
0924 07:14:53.194559 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:53.194569 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:53.194628 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:53.194752 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:53.194753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:53.194910 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:53.194962 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:53.194964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.195667 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.195668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:53.196328 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:53.196331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.196641 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.196641 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:53.196678 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:53.197124 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:53.197128 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:53.198701 (+ 1573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
W20260924 07:14:53.198850 19934 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:53.194513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:37953 (local address 127.10.130.254:34947)
0924 07:14:53.194618 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:53.194619 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:53.194628 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:53.194685 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:53.194686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:53.194696 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:53.194742 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:53.195018 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.195594 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.196394 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.196588 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.196638 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:53.198780 (+ 2142us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.193:37953: BlockingRecv error: recv got EOF from 127.10.130.193:37953 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":56,"threads_started":1}
Sep 24 07:14:53 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 07:14:53 dist-test-slave-7q7m krb5kdc[18875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.130.194@KRBTEST.COM for oryx/127.10.130.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 07:14:53.296816 19935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:53.292625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34947 (local address 127.10.130.194:48477)
0924 07:14:53.292735 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:53.292747 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:53.292809 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:53.292933 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:53.292934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:53.293092 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:53.293136 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:53.293138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.293817 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.293818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:53.294353 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:53.294355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.294689 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.294690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:53.294741 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:53.295146 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:53.295150 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:53.296742 (+ 1592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.130.254:34947: Server oryx/127.10.130.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
W20260924 07:14:53.296892 19934 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:53.292710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:48477 (local address 127.10.130.254:34947)
0924 07:14:53.292765 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:53.292767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:53.292790 (+ 23us) server_negotiation.cc:407] Connection header received
0924 07:14:53.292858 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:53.292859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:53.292872 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:53.292916 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:53.293201 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.293739 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.294436 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.294632 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.294686 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:53.296841 (+ 2155us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.130.194:48477: BlockingRecv error: recv got EOF from 127.10.130.194:48477 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 07:14:53.354650 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 19459
I20260924 07:14:53.358515 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 19610
I20260924 07:14:53.362140 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 19744
I20260924 07:14:53.365536 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 19388
2026-09-24T07:14:53Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17186 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-24T07:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:53Z Disabled control of system clock
I20260924 07:14:53.390218 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:36131
--webserver_interface=127.10.130.254
--webserver_port=0
--builtin_ntp_servers=127.10.130.212:42215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:36131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 07:14:53.466969 19939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:53.467150 19939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:53.467175 19939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:53.468334 19939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:53.468372 19939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:53.468389 19939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:53.468405 19939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:53.468420 19939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:53.469695 19939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:42215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:36131
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:36131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:53.469923 19939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:53.470165 19939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:53.474506 19945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:53.474649 19939 server_base.cc:1034] running on GCE node
W20260924 07:14:53.474519 19944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:53.474526 19947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:53.474936 19939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:53.475208 19939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:53.476348 19939 hybrid_clock.cc:648] HybridClock initialized: now 1790234093476333 us; error 26 us; skew 500 ppm
I20260924 07:14:53.477421 19939 webserver.cc:460] Webserver started at http://127.10.130.254:42709/ using document root <none> and password file <none>
I20260924 07:14:53.477629 19939 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:53.477675 19939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:53.477789 19939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:53.478730 19939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "38e168b31ac54a8ab041b69c2b509842"
format_stamp: "Formatted at 2026-09-24 07:14:53 on dist-test-slave-7q7m"
server_key: "2396b9ea34034dbe4adeb40df9c94214"
server_key_iv: "90ee18ea26ea80490a66de01559e5904"
server_key_version: "encryptionkey@0"
I20260924 07:14:53.479035 19939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "38e168b31ac54a8ab041b69c2b509842"
format_stamp: "Formatted at 2026-09-24 07:14:53 on dist-test-slave-7q7m"
server_key: "2396b9ea34034dbe4adeb40df9c94214"
server_key_iv: "90ee18ea26ea80490a66de01559e5904"
server_key_version: "encryptionkey@0"
I20260924 07:14:53.480191 19939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:53.480801 19953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:53.480937 19939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:53.481006 19939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "38e168b31ac54a8ab041b69c2b509842"
format_stamp: "Formatted at 2026-09-24 07:14:53 on dist-test-slave-7q7m"
server_key: "2396b9ea34034dbe4adeb40df9c94214"
server_key_iv: "90ee18ea26ea80490a66de01559e5904"
server_key_version: "encryptionkey@0"
I20260924 07:14:53.481069 19939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:53.488893 19939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:53.489142 19939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:53.489256 19939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:53.492861 19939 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:36131
I20260924 07:14:53.492892 20005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:36131 every 8 connection(s)
I20260924 07:14:53.493189 19939 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:53.493511 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 19939
I20260924 07:14:53.493600 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:53.493767 10763 external_mini_cluster.cc:1428] Setting key 09bc93c01e29679460f49e27d3e3683e
I20260924 07:14:53.494129 20006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:53.497166 20006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842: Bootstrap starting.
I20260924 07:14:53.497706 20006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:53.497957 20006 log.cc:824] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:53.498559 20006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842: No bootstrap required, opened a new log
I20260924 07:14:53.499048 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:53.494206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40210 (local address 127.10.130.254:36131)
0924 07:14:53.494413 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:53.494422 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:53.494452 (+ 30us) server_negotiation.cc:407] Connection header received
0924 07:14:53.494888 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:53.494897 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:53.494975 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:53.495131 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:53.495262 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.496569 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.497313 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.497710 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.497817 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:53.498437 (+ 620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:53.498441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:53.498441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:53.498599 (+ 158us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:53.498630 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:53.498643 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:53.498702 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:53.498789 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 07:14:53.498851 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260924 07:14:53.500612 20006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e168b31ac54a8ab041b69c2b509842" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36131 } }
I20260924 07:14:53.500741 20006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:53.500772 20006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38e168b31ac54a8ab041b69c2b509842, State: Initialized, Role: FOLLOWER
I20260924 07:14:53.500875 20006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e168b31ac54a8ab041b69c2b509842" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36131 } }
I20260924 07:14:53.500952 20006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:53.500988 20006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:53.501034 20006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:53.501513 20006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e168b31ac54a8ab041b69c2b509842" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36131 } }
I20260924 07:14:53.501601 20006 leader_election.cc:302] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38e168b31ac54a8ab041b69c2b509842; no voters:
I20260924 07:14:53.501746 20006 leader_election.cc:288] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:53.501792 20011 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:53.501969 20011 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [term 1 LEADER]: Becoming Leader. State: Replica: 38e168b31ac54a8ab041b69c2b509842, State: Running, Role: LEADER
I20260924 07:14:53.501989 20006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:53.502056 20011 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e168b31ac54a8ab041b69c2b509842" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36131 } }
I20260924 07:14:53.502388 20013 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38e168b31ac54a8ab041b69c2b509842. Latest consensus state: current_term: 1 leader_uuid: "38e168b31ac54a8ab041b69c2b509842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e168b31ac54a8ab041b69c2b509842" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36131 } } }
I20260924 07:14:53.502465 20013 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:53.502640 20012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38e168b31ac54a8ab041b69c2b509842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38e168b31ac54a8ab041b69c2b509842" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36131 } } }
I20260924 07:14:53.502668 20019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:53.502727 20012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842 [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:53.503199 20019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:53.504585 20019 catalog_manager.cc:1347] Generated new cluster ID: f56c77b011f94225a1e7b398d450b4b4
I20260924 07:14:53.504632 20019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:53.523792 20019 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:53.524555 20019 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:53.529538 20019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38e168b31ac54a8ab041b69c2b509842: Generated new TSK 0
I20260924 07:14:53.529711 20019 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:14:53.534401 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:36131
--builtin_ntp_servers=127.10.130.212:42215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 07:14:53.612143 20030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:53.612334 20030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:53.612353 20030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:53.613462 20030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:53.613502 20030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:53.613524 20030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:53.614781 20030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:42215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:36131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:53.614984 20030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:53.615260 20030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 07:14:53.620725 20030 server_base.cc:1034] running on GCE node
W20260924 07:14:53.620698 20036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:53.620796 20038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:53.620692 20035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:53.621090 20030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:53.621311 20030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:53.622447 20030 hybrid_clock.cc:648] HybridClock initialized: now 1790234093622435 us; error 29 us; skew 500 ppm
I20260924 07:14:53.623569 20030 webserver.cc:460] Webserver started at http://127.10.130.193:43161/ using document root <none> and password file <none>
I20260924 07:14:53.623782 20030 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:53.623831 20030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:53.623948 20030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:53.624959 20030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "35dc213e98de4becac82d48abde5adcd"
format_stamp: "Formatted at 2026-09-24 07:14:53 on dist-test-slave-7q7m"
server_key: "bbf10bb6b306b795d7412ea3df388497"
server_key_iv: "c2ab301ed8b7711b79776908fb615ca4"
server_key_version: "encryptionkey@0"
I20260924 07:14:53.625277 20030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "35dc213e98de4becac82d48abde5adcd"
format_stamp: "Formatted at 2026-09-24 07:14:53 on dist-test-slave-7q7m"
server_key: "bbf10bb6b306b795d7412ea3df388497"
server_key_iv: "c2ab301ed8b7711b79776908fb615ca4"
server_key_version: "encryptionkey@0"
I20260924 07:14:53.626502 20030 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:53.627105 20044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:53.627272 20030 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:53.627345 20030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "35dc213e98de4becac82d48abde5adcd"
format_stamp: "Formatted at 2026-09-24 07:14:53 on dist-test-slave-7q7m"
server_key: "bbf10bb6b306b795d7412ea3df388497"
server_key_iv: "c2ab301ed8b7711b79776908fb615ca4"
server_key_version: "encryptionkey@0"
I20260924 07:14:53.627405 20030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:53.642901 20030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:53.643267 20030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:53.643438 20030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:53.643649 20030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:53.644024 20030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:53.644065 20030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:53.644097 20030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:53.644125 20030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:53.649891 20030 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:43885
I20260924 07:14:53.649924 20157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:43885 every 8 connection(s)
I20260924 07:14:53.650220 20030 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:14:53.655010 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:53.650883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:42071 (local address 127.10.130.254:36131)
0924 07:14:53.650946 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:53.650948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:53.651202 (+ 254us) server_negotiation.cc:407] Connection header received
0924 07:14:53.651677 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:53.651678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:53.651692 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:53.651736 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:53.652421 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.652925 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.653867 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.654059 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.654112 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:53.654791 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:53.654792 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:53.654793 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:53.654797 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:53.654809 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:53.654812 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:53.654848 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:53.654921 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 07:14:53.654952 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 07:14:53.655196 20160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:53.650866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:36131 (local address 127.10.130.193:42071)
0924 07:14:53.651071 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:53.651102 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:53.651582 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:53.651801 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:53.651805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:53.651813 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:53.652319 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:53.652325 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.653004 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.653005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:53.653788 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:53.653790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.654122 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.654123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:53.654226 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:53.654674 (+ 448us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:53.654675 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:53.654701 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:53.654723 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:53.654857 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:53.654859 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:53.654861 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:14:53.654919 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 07:14:53.654995 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260924 07:14:53.655694 20158 heartbeater.cc:347] Connected to a master server at 127.10.130.254:36131
I20260924 07:14:53.655774 20158 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:53.655917 20158 heartbeater.cc:511] Master 127.10.130.254:36131 requested a full tablet report, sending...
I20260924 07:14:53.656409 19970 ts_manager.cc:194] Registered new tserver with Master: 35dc213e98de4becac82d48abde5adcd (127.10.130.193:43885)
I20260924 07:14:53.657008 19970 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.130.193:42071
I20260924 07:14:53.659060 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 20030
I20260924 07:14:53.659139 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:53.659308 10763 external_mini_cluster.cc:1428] Setting key 91db219c992c9dbffd6b0489f512aebd
I20260924 07:14:53.660085 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:36131
--builtin_ntp_servers=127.10.130.212:42215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 07:14:53.737536 20161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:53.737722 20161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:53.737751 20161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:53.738842 20161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:53.738878 20161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:53.738909 20161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:53.740221 20161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:42215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:36131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:53.740443 20161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:53.740698 20161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:53.745234 20169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:53.745245 20167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:53.745226 20166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:53.745362 20161 server_base.cc:1034] running on GCE node
I20260924 07:14:53.745608 20161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:53.745828 20161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:53.746959 20161 hybrid_clock.cc:648] HybridClock initialized: now 1790234093746946 us; error 29 us; skew 500 ppm
I20260924 07:14:53.748160 20161 webserver.cc:460] Webserver started at http://127.10.130.194:33063/ using document root <none> and password file <none>
I20260924 07:14:53.748365 20161 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:53.748411 20161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:53.748531 20161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:53.749533 20161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "1cbb371fd20249ba9fa9c22f1d75fefe"
format_stamp: "Formatted at 2026-09-24 07:14:53 on dist-test-slave-7q7m"
server_key: "ff5c86a24da3acee63f8094f9c68e816"
server_key_iv: "cc4820cb29d2cd3b00f837e0371591d6"
server_key_version: "encryptionkey@0"
I20260924 07:14:53.749836 20161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "1cbb371fd20249ba9fa9c22f1d75fefe"
format_stamp: "Formatted at 2026-09-24 07:14:53 on dist-test-slave-7q7m"
server_key: "ff5c86a24da3acee63f8094f9c68e816"
server_key_iv: "cc4820cb29d2cd3b00f837e0371591d6"
server_key_version: "encryptionkey@0"
I20260924 07:14:53.751006 20161 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:14:53.751600 20175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:53.751727 20161 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:53.751803 20161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "1cbb371fd20249ba9fa9c22f1d75fefe"
format_stamp: "Formatted at 2026-09-24 07:14:53 on dist-test-slave-7q7m"
server_key: "ff5c86a24da3acee63f8094f9c68e816"
server_key_iv: "cc4820cb29d2cd3b00f837e0371591d6"
server_key_version: "encryptionkey@0"
I20260924 07:14:53.751863 20161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:53.760562 20161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:53.760797 20161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:53.760914 20161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:53.761082 20161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:53.761348 20161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:53.761377 20161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:53.761404 20161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:53.761420 20161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:53.766584 20161 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:42165
I20260924 07:14:53.766636 20288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:42165 every 8 connection(s)
I20260924 07:14:53.766908 20161 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:14:53.771579 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:53.767501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:38849 (local address 127.10.130.254:36131)
0924 07:14:53.767544 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:53.767546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:53.767756 (+ 210us) server_negotiation.cc:407] Connection header received
0924 07:14:53.768282 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:53.768283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:53.768295 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:53.768347 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:53.768977 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.769465 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.770402 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.770595 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.770650 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:53.771334 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:53.771336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:53.771336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:53.771340 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:53.771356 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:53.771360 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:53.771403 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:53.771474 (+ 71us) server_negotiation.cc:299] Negotiation successful
0924 07:14:53.771498 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:14:53.771759 20291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:53.767479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:36131 (local address 127.10.130.194:38849)
0924 07:14:53.767648 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:53.767678 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:53.768193 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:53.768413 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:53.768417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:53.768427 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:53.768891 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:53.768896 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.769547 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.769548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:53.770321 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:53.770323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.770664 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.770665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:53.770785 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:53.771214 (+ 429us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:53.771214 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:53.771241 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:53.771261 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:53.771409 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:53.771414 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:53.771417 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:53.771473 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 07:14:53.771581 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260924 07:14:53.772192 20289 heartbeater.cc:347] Connected to a master server at 127.10.130.254:36131
I20260924 07:14:53.772270 20289 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:53.772445 20289 heartbeater.cc:511] Master 127.10.130.254:36131 requested a full tablet report, sending...
I20260924 07:14:53.772812 19970 ts_manager.cc:194] Registered new tserver with Master: 1cbb371fd20249ba9fa9c22f1d75fefe (127.10.130.194:42165)
I20260924 07:14:53.773281 19970 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.130.194:38849
I20260924 07:14:53.773901 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 20161
I20260924 07:14:53.773984 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:53.774152 10763 external_mini_cluster.cc:1428] Setting key d576ac88678986c449d22365b642c23c
I20260924 07:14:53.774915 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:36131
--builtin_ntp_servers=127.10.130.212:42215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 07:14:53.852002 20292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:53.852212 20292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:53.852231 20292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:53.853353 20292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:53.853391 20292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:53.853417 20292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:53.854779 20292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:42215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:36131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:53.854992 20292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:53.855280 20292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:53.859958 20300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:53.860036 20297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:53.860028 20298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:53.860235 20292 server_base.cc:1034] running on GCE node
I20260924 07:14:53.860430 20292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:53.860668 20292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:53.861797 20292 hybrid_clock.cc:648] HybridClock initialized: now 1790234093861787 us; error 31 us; skew 500 ppm
I20260924 07:14:53.862936 20292 webserver.cc:460] Webserver started at http://127.10.130.195:39449/ using document root <none> and password file <none>
I20260924 07:14:53.863155 20292 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:53.863197 20292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:53.863322 20292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:53.864401 20292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "8680b24c8be940d5b9e194a6346ca011"
format_stamp: "Formatted at 2026-09-24 07:14:53 on dist-test-slave-7q7m"
server_key: "e8aa01f7c37cb594436a70c8d9a3dc24"
server_key_iv: "f08d3e57164a2d216383b4aecb519c0b"
server_key_version: "encryptionkey@0"
I20260924 07:14:53.864733 20292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "8680b24c8be940d5b9e194a6346ca011"
format_stamp: "Formatted at 2026-09-24 07:14:53 on dist-test-slave-7q7m"
server_key: "e8aa01f7c37cb594436a70c8d9a3dc24"
server_key_iv: "f08d3e57164a2d216383b4aecb519c0b"
server_key_version: "encryptionkey@0"
I20260924 07:14:53.865876 20292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:14:53.866498 20306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:53.866628 20292 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:53.866704 20292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "8680b24c8be940d5b9e194a6346ca011"
format_stamp: "Formatted at 2026-09-24 07:14:53 on dist-test-slave-7q7m"
server_key: "e8aa01f7c37cb594436a70c8d9a3dc24"
server_key_iv: "f08d3e57164a2d216383b4aecb519c0b"
server_key_version: "encryptionkey@0"
I20260924 07:14:53.866771 20292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:53.875507 20292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:53.875767 20292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:53.875880 20292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:53.876094 20292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:53.876397 20292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:53.876434 20292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:53.876456 20292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:53.876478 20292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:53.881770 20292 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:45219
I20260924 07:14:53.881819 20419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:45219 every 8 connection(s)
I20260924 07:14:53.882103 20292 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:53.886699 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:53.882737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:32835 (local address 127.10.130.254:36131)
0924 07:14:53.882791 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:53.882793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:53.882991 (+ 198us) server_negotiation.cc:407] Connection header received
0924 07:14:53.883488 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:53.883489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:53.883500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:53.883537 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:53.884239 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.884721 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.885611 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.885787 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.885834 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:53.886501 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:53.886502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:53.886502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:53.886506 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:53.886520 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:53.886523 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:53.886562 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:53.886623 (+ 61us) server_negotiation.cc:299] Negotiation successful
0924 07:14:53.886642 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 07:14:53.886891 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:53.882716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:36131 (local address 127.10.130.195:32835)
0924 07:14:53.882883 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:53.882922 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:53.883391 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:53.883605 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:53.883608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:53.883616 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:53.884176 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:53.884182 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.884784 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.884784 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:53.885539 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:53.885541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:53.885843 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:53.885843 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:53.885943 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:53.886369 (+ 426us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:53.886369 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:53.886396 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:53.886417 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:53.886576 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:53.886578 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:53.886581 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:53.886625 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 07:14:53.886697 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260924 07:14:53.887285 20420 heartbeater.cc:347] Connected to a master server at 127.10.130.254:36131
I20260924 07:14:53.887364 20420 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:53.887526 20420 heartbeater.cc:511] Master 127.10.130.254:36131 requested a full tablet report, sending...
I20260924 07:14:53.887889 19970 ts_manager.cc:194] Registered new tserver with Master: 8680b24c8be940d5b9e194a6346ca011 (127.10.130.195:45219)
I20260924 07:14:53.888262 19970 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.130.195:32835
I20260924 07:14:53.889021 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 20292
I20260924 07:14:53.889104 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:53.889288 10763 external_mini_cluster.cc:1428] Setting key c2802bdde9569fbe69405ae2f389f60e
I20260924 07:14:53.890064 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 07:14:53.891723 20009 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:53.891308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40226 (local address 127.10.130.254:36131)
0924 07:14:53.891363 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:53.891365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:53.891463 (+ 98us) server_negotiation.cc:407] Connection header received
0924 07:14:53.891527 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:53.891528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:53.891536 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:53.891580 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:53.891680 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40226: BlockingRecv error: recv got EOF from 127.0.0.1:40226 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 07:14:53.891839 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 20030
I20260924 07:14:53.896304 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 20161
I20260924 07:14:53.899930 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 20292
I20260924 07:14:53.903249 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 19939
2026-09-24T07:14:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (536 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-24T07:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:53Z Disabled control of system clock
I20260924 07:14:53.926927 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:34649
--webserver_interface=127.10.130.254
--webserver_port=0
--builtin_ntp_servers=127.10.130.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:34649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 07:14:54.003585 20431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:54.003779 20431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:54.003799 20431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:54.004895 20431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:54.004930 20431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:54.004941 20431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:54.004952 20431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:54.004972 20431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:54.006212 20431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:34649
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:34649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:54.006402 20431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:54.006626 20431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:54.011081 20436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:54.011204 20437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:54.011134 20439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:54.011308 20431 server_base.cc:1034] running on GCE node
I20260924 07:14:54.011513 20431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:54.011766 20431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:54.012895 20431 hybrid_clock.cc:648] HybridClock initialized: now 1790234094012878 us; error 31 us; skew 500 ppm
I20260924 07:14:54.013952 20431 webserver.cc:460] Webserver started at http://127.10.130.254:42505/ using document root <none> and password file <none>
I20260924 07:14:54.014125 20431 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:54.014163 20431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:54.014250 20431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:54.015219 20431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "d471c82c559e4d63a50e3c6e08f4c54a"
format_stamp: "Formatted at 2026-09-24 07:14:54 on dist-test-slave-7q7m"
server_key: "930ed974a5faf2cf941ce768f2ce775d"
server_key_iv: "754b2aeabcf02087254daf9681f6e2e8"
server_key_version: "encryptionkey@0"
I20260924 07:14:54.015508 20431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "d471c82c559e4d63a50e3c6e08f4c54a"
format_stamp: "Formatted at 2026-09-24 07:14:54 on dist-test-slave-7q7m"
server_key: "930ed974a5faf2cf941ce768f2ce775d"
server_key_iv: "754b2aeabcf02087254daf9681f6e2e8"
server_key_version: "encryptionkey@0"
I20260924 07:14:54.016647 20431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:14:54.017241 20445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:54.017364 20431 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:54.017428 20431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "d471c82c559e4d63a50e3c6e08f4c54a"
format_stamp: "Formatted at 2026-09-24 07:14:54 on dist-test-slave-7q7m"
server_key: "930ed974a5faf2cf941ce768f2ce775d"
server_key_iv: "754b2aeabcf02087254daf9681f6e2e8"
server_key_version: "encryptionkey@0"
I20260924 07:14:54.017483 20431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:54.020601 20431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:54.020800 20431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:54.020884 20431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:54.024353 20431 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:34649
I20260924 07:14:54.024385 20497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:34649 every 8 connection(s)
I20260924 07:14:54.024678 20431 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:54.025247 20498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:54.027801 20498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a: Bootstrap starting.
I20260924 07:14:54.028359 20498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:54.028618 20498 log.cc:824] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:54.029246 20498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a: No bootstrap required, opened a new log
I20260924 07:14:54.030620 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 20431
I20260924 07:14:54.030700 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:54.030869 10763 external_mini_cluster.cc:1428] Setting key b924f35e8fd0d8e5be36cd42d8e45d77
I20260924 07:14:54.030789 20498 raft_consensus.cc:348] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d471c82c559e4d63a50e3c6e08f4c54a" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34649 } }
I20260924 07:14:54.030918 20498 raft_consensus.cc:374] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:54.030959 20498 raft_consensus.cc:729] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d471c82c559e4d63a50e3c6e08f4c54a, State: Initialized, Role: FOLLOWER
I20260924 07:14:54.031108 20498 consensus_queue.cc:260] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d471c82c559e4d63a50e3c6e08f4c54a" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34649 } }
I20260924 07:14:54.031193 20498 raft_consensus.cc:388] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:54.031224 20498 raft_consensus.cc:482] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:54.031301 20498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:54.032032 20498 raft_consensus.cc:504] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d471c82c559e4d63a50e3c6e08f4c54a" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34649 } }
I20260924 07:14:54.032155 20498 leader_election.cc:302] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d471c82c559e4d63a50e3c6e08f4c54a; no voters:
I20260924 07:14:54.032302 20498 leader_election.cc:288] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:54.032344 20503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:54.032562 20503 raft_consensus.cc:686] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [term 1 LEADER]: Becoming Leader. State: Replica: d471c82c559e4d63a50e3c6e08f4c54a, State: Running, Role: LEADER
I20260924 07:14:54.032644 20498 sys_catalog.cc:564] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:54.032708 20503 consensus_queue.cc:237] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d471c82c559e4d63a50e3c6e08f4c54a" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34649 } }
I20260924 07:14:54.033041 20503 sys_catalog.cc:455] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d471c82c559e4d63a50e3c6e08f4c54a. Latest consensus state: current_term: 1 leader_uuid: "d471c82c559e4d63a50e3c6e08f4c54a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d471c82c559e4d63a50e3c6e08f4c54a" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34649 } } }
I20260924 07:14:54.033125 20503 sys_catalog.cc:458] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:54.033300 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:54.031375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56948 (local address 127.10.130.254:34649)
0924 07:14:54.031607 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:54.031616 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:54.031648 (+ 32us) server_negotiation.cc:407] Connection header received
0924 07:14:54.031768 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:54.031775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:54.031822 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:54.031972 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:54.032674 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:54.032678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:54.032678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:54.032682 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:54.032720 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:54.032734 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:54.032947 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:54.032977 (+ 30us) server_negotiation.cc:299] Negotiation successful
0924 07:14:54.033034 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260924 07:14:54.033325 20510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:54.033502 20505 sys_catalog.cc:455] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d471c82c559e4d63a50e3c6e08f4c54a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d471c82c559e4d63a50e3c6e08f4c54a" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 34649 } } }
I20260924 07:14:54.033622 20505 sys_catalog.cc:458] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:54.033818 20510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:54.035211 20510 catalog_manager.cc:1347] Generated new cluster ID: c40406483b874c0dba14250940fd8574
I20260924 07:14:54.035265 20510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:14:54.044023 20510 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:54.044190 20510 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:54.048921 20510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d471c82c559e4d63a50e3c6e08f4c54a: Generated new TSK 0
I20260924 07:14:54.049088 20510 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:14:54.054018 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:34649
--builtin_ntp_servers=127.10.130.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 07:14:54.150031 20522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:54.150221 20522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:54.150242 20522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:54.151391 20522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:54.151430 20522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:54.151463 20522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:54.152840 20522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:34649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:54.153049 20522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:54.153321 20522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:54.157964 20528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:54.157974 20527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:54.158013 20530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:54.158406 20522 server_base.cc:1034] running on GCE node
I20260924 07:14:54.158592 20522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:54.158813 20522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:54.159974 20522 hybrid_clock.cc:648] HybridClock initialized: now 1790234094159960 us; error 31 us; skew 500 ppm
I20260924 07:14:54.161219 20522 webserver.cc:460] Webserver started at http://127.10.130.193:41999/ using document root <none> and password file <none>
I20260924 07:14:54.161453 20522 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:54.161504 20522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:54.161633 20522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:54.162662 20522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "9e3582c3eeff464bb7aac6f409cc5473"
format_stamp: "Formatted at 2026-09-24 07:14:54 on dist-test-slave-7q7m"
server_key: "8c068f698192985cb41b5f6fb2885fda"
server_key_iv: "c0d40bd77b841407675f68621a042b6a"
server_key_version: "encryptionkey@0"
I20260924 07:14:54.162992 20522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "9e3582c3eeff464bb7aac6f409cc5473"
format_stamp: "Formatted at 2026-09-24 07:14:54 on dist-test-slave-7q7m"
server_key: "8c068f698192985cb41b5f6fb2885fda"
server_key_iv: "c0d40bd77b841407675f68621a042b6a"
server_key_version: "encryptionkey@0"
I20260924 07:14:54.164228 20522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:14:54.164927 20536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:54.165081 20522 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:54.165155 20522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "9e3582c3eeff464bb7aac6f409cc5473"
format_stamp: "Formatted at 2026-09-24 07:14:54 on dist-test-slave-7q7m"
server_key: "8c068f698192985cb41b5f6fb2885fda"
server_key_iv: "c0d40bd77b841407675f68621a042b6a"
server_key_version: "encryptionkey@0"
I20260924 07:14:54.165221 20522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:54.168406 20522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:54.168669 20522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:54.168788 20522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:54.168982 20522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:54.169250 20522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:54.169281 20522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:54.169307 20522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:54.169332 20522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:54.174635 20522 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:34655
I20260924 07:14:54.174682 20649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:34655 every 8 connection(s)
I20260924 07:14:54.174997 20522 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:14:54.178043 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:54.175741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:42115 (local address 127.10.130.254:34649)
0924 07:14:54.175803 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:54.175805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:54.176069 (+ 264us) server_negotiation.cc:407] Connection header received
0924 07:14:54.176637 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:54.176639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:54.176644 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:54.176686 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:54.177852 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:54.177853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:54.177853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:54.177856 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:54.177869 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:54.177873 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:54.177915 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:54.177980 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 07:14:54.177991 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 07:14:54.178467 20652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:54.175729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34649 (local address 127.10.130.193:42115)
0924 07:14:54.175916 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:54.175958 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:54.176537 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:54.176757 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:54.176761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:54.176770 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:54.177704 (+ 934us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:54.177705 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:54.177742 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:54.177774 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:54.177916 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:54.177920 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:54.177923 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:54.177986 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 07:14:54.178047 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260924 07:14:54.178829 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 20522
I20260924 07:14:54.178916 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:54.179069 20650 heartbeater.cc:347] Connected to a master server at 127.10.130.254:34649
I20260924 07:14:54.179096 10763 external_mini_cluster.cc:1428] Setting key a62ca543abb8b2769e31754598a275f0
I20260924 07:14:54.179150 20650 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:54.179288 20650 heartbeater.cc:511] Master 127.10.130.254:34649 requested a full tablet report, sending...
I20260924 07:14:54.179759 20461 ts_manager.cc:194] Registered new tserver with Master: 9e3582c3eeff464bb7aac6f409cc5473 (127.10.130.193:34655)
I20260924 07:14:54.179929 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:34649
--builtin_ntp_servers=127.10.130.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 07:14:54.256664 20653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:54.256845 20653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:54.256865 20653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:54.257982 20653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:54.258018 20653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:54.258042 20653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:54.259306 20653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:34649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:54.259506 20653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:54.259766 20653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:54.264250 20661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:54.264242 20658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:54.264315 20659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:54.264456 20653 server_base.cc:1034] running on GCE node
I20260924 07:14:54.264649 20653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:54.264871 20653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:54.266001 20653 hybrid_clock.cc:648] HybridClock initialized: now 1790234094265984 us; error 32 us; skew 500 ppm
I20260924 07:14:54.269830 20653 webserver.cc:460] Webserver started at http://127.10.130.194:39167/ using document root <none> and password file <none>
I20260924 07:14:54.270030 20653 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:54.270087 20653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:54.270210 20653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:54.271221 20653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "658c33b14acd4f3baf6ff5fb0bd59083"
format_stamp: "Formatted at 2026-09-24 07:14:54 on dist-test-slave-7q7m"
server_key: "08b1e4f96918a149e4374e7284ec3e74"
server_key_iv: "d108618cd0106d94864a513fc03323d1"
server_key_version: "encryptionkey@0"
I20260924 07:14:54.271598 20653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "658c33b14acd4f3baf6ff5fb0bd59083"
format_stamp: "Formatted at 2026-09-24 07:14:54 on dist-test-slave-7q7m"
server_key: "08b1e4f96918a149e4374e7284ec3e74"
server_key_iv: "d108618cd0106d94864a513fc03323d1"
server_key_version: "encryptionkey@0"
I20260924 07:14:54.272794 20653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:14:54.273403 20667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:54.273553 20653 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:54.273626 20653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "658c33b14acd4f3baf6ff5fb0bd59083"
format_stamp: "Formatted at 2026-09-24 07:14:54 on dist-test-slave-7q7m"
server_key: "08b1e4f96918a149e4374e7284ec3e74"
server_key_iv: "d108618cd0106d94864a513fc03323d1"
server_key_version: "encryptionkey@0"
I20260924 07:14:54.273693 20653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:54.276880 20653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:54.277109 20653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:54.277216 20653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:54.277385 20653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:54.277652 20653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:54.277681 20653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:54.277707 20653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:54.277730 20653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:54.283119 20653 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:38503
I20260924 07:14:54.283481 20653 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:14:54.283773 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 20653
I20260924 07:14:54.283859 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:54.284077 10763 external_mini_cluster.cc:1428] Setting key 229bced343328b63ce1d6458aec6145e
I20260924 07:14:54.283968 20780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:38503 every 8 connection(s)
I20260924 07:14:54.285163 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:34649
--builtin_ntp_servers=127.10.130.212:45351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260924 07:14:54.286808 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:54.284399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:37431 (local address 127.10.130.254:34649)
0924 07:14:54.284448 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:54.284450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:54.284576 (+ 126us) server_negotiation.cc:407] Connection header received
0924 07:14:54.285141 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:54.285142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:54.285147 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:54.285183 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:54.286550 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:54.286551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:54.286552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:54.286556 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:54.286581 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:54.286586 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:54.286647 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:54.286723 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 07:14:54.286737 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 07:14:54.286933 20783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:54.284260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34649 (local address 127.10.130.194:37431)
0924 07:14:54.284462 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:54.284503 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:54.285042 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:54.285382 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:54.285386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:54.285396 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:54.286377 (+ 981us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:54.286377 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:54.286416 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:54.286452 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:54.286652 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:54.286656 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:54.286658 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:14:54.286714 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 07:14:54.286760 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":75,"threads_started":1}
I20260924 07:14:54.287381 20781 heartbeater.cc:347] Connected to a master server at 127.10.130.254:34649
I20260924 07:14:54.287473 20781 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:54.287638 20781 heartbeater.cc:511] Master 127.10.130.254:34649 requested a full tablet report, sending...
I20260924 07:14:54.287968 20461 ts_manager.cc:194] Registered new tserver with Master: 658c33b14acd4f3baf6ff5fb0bd59083 (127.10.130.194:38503)
W20260924 07:14:54.363449 20784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:54.363631 20784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:54.363651 20784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:54.364724 20784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:54.364759 20784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:54.364782 20784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:54.366031 20784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:45351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:34649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:54.366209 20784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:54.366441 20784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:54.370952 20789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:54.370968 20792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:54.371038 20784 server_base.cc:1034] running on GCE node
W20260924 07:14:54.370975 20790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:54.371354 20784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:54.371588 20784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:54.372711 20784 hybrid_clock.cc:648] HybridClock initialized: now 1790234094372705 us; error 28 us; skew 500 ppm
I20260924 07:14:54.373781 20784 webserver.cc:460] Webserver started at http://127.10.130.195:42335/ using document root <none> and password file <none>
I20260924 07:14:54.373986 20784 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:54.374028 20784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:54.374146 20784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:54.375125 20784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "776c405c0efe4deeae61f66aaaf1b873"
format_stamp: "Formatted at 2026-09-24 07:14:54 on dist-test-slave-7q7m"
server_key: "c6989aa924ce93b670eac2a0ab2b6972"
server_key_iv: "87a7c3bc6de7f7956bb26f5741505821"
server_key_version: "encryptionkey@0"
I20260924 07:14:54.375420 20784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "776c405c0efe4deeae61f66aaaf1b873"
format_stamp: "Formatted at 2026-09-24 07:14:54 on dist-test-slave-7q7m"
server_key: "c6989aa924ce93b670eac2a0ab2b6972"
server_key_iv: "87a7c3bc6de7f7956bb26f5741505821"
server_key_version: "encryptionkey@0"
I20260924 07:14:54.376624 20784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:14:54.377274 20798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:54.377403 20784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:54.377487 20784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "776c405c0efe4deeae61f66aaaf1b873"
format_stamp: "Formatted at 2026-09-24 07:14:54 on dist-test-slave-7q7m"
server_key: "c6989aa924ce93b670eac2a0ab2b6972"
server_key_iv: "87a7c3bc6de7f7956bb26f5741505821"
server_key_version: "encryptionkey@0"
I20260924 07:14:54.377544 20784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:54.380650 20784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:54.380882 20784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:54.380990 20784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:54.381160 20784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:54.381414 20784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:54.381445 20784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:54.381470 20784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:54.381505 20784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:54.386493 20784 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:46711
I20260924 07:14:54.386544 20911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:46711 every 8 connection(s)
I20260924 07:14:54.386793 20784 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:14:54.389324 20502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:54.387425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:43127 (local address 127.10.130.254:34649)
0924 07:14:54.387477 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:54.387479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:54.387695 (+ 216us) server_negotiation.cc:407] Connection header received
0924 07:14:54.388178 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:54.388178 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:54.388183 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:54.388218 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:54.389157 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:54.389158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:54.389159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:14:54.389162 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:54.389174 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:14:54.389178 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:54.389219 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:54.389262 (+ 43us) server_negotiation.cc:299] Negotiation successful
0924 07:14:54.389273 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 07:14:54.389509 20914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:54.387414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:34649 (local address 127.10.130.195:43127)
0924 07:14:54.387577 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:54.387613 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:54.388082 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:54.388275 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:54.388278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:54.388286 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:54.389038 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:14:54.389039 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:54.389065 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:14:54.389089 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:54.389218 (+ 129us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:54.389220 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:54.389222 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:14:54.389267 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 07:14:54.389308 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
I20260924 07:14:54.389902 20912 heartbeater.cc:347] Connected to a master server at 127.10.130.254:34649
I20260924 07:14:54.389974 20912 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:54.390120 20912 heartbeater.cc:511] Master 127.10.130.254:34649 requested a full tablet report, sending...
I20260924 07:14:54.390184 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 20784
I20260924 07:14:54.390261 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:54.390455 10763 external_mini_cluster.cc:1428] Setting key ecb2b0830ee4b99c5ac0e88a81014358
I20260924 07:14:54.390475 20461 ts_manager.cc:194] Registered new tserver with Master: 776c405c0efe4deeae61f66aaaf1b873 (127.10.130.195:46711)
I20260924 07:14:54.391139 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 07:14:54.392705 20502 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:14:54.392317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56960 (local address 127.10.130.254:34649)
0924 07:14:54.392365 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:54.392367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:54.392441 (+ 74us) server_negotiation.cc:407] Connection header received
0924 07:14:54.392508 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:54.392509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:54.392513 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:54.392551 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:54.392651 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56960: BlockingRecv error: recv got EOF from 127.0.0.1:56960 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 07:14:54.392761 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 20522
I20260924 07:14:54.397321 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 20653
I20260924 07:14:54.401054 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 20784
I20260924 07:14:54.405089 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 20431
2026-09-24T07:14:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (501 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:54 dist-test-slave-7q7m krb5kdc[20922](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:54 dist-test-slave-7q7m krb5kdc[20922](info): set up 2 sockets
Sep 24 07:14:54 dist-test-slave-7q7m krb5kdc[20922](info): commencing operation
krb5kdc: starting...
W20260924 07:14:55.348374 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.925s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:55 dist-test-slave-7q7m krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:14:55.450734 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:36453
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:36453
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:55.529245 20935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:55.529451 20935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:55.529481 20935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:55.530596 20935 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:14:55.530634 20935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:55.530653 20935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:55.530668 20935 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:14:55.530684 20935 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:14:55.532177 20935 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:36453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:36453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:55.532423 20935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:55.532619 20935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:55.537106 20943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:55.537103 20940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:55.537082 20941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:55.537197 20935 server_base.cc:1034] running on GCE node
I20260924 07:14:55.537472 20935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:55.537740 20935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:55.538878 20935 hybrid_clock.cc:648] HybridClock initialized: now 1790234095538862 us; error 28 us; skew 500 ppm
Sep 24 07:14:55 dist-test-slave-7q7m krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234095, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:55.541132 20935 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:14:55.541517 20935 webserver.cc:460] Webserver started at http://127.10.130.254:33661/ using document root <none> and password file <none>
I20260924 07:14:55.541721 20935 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:55.541769 20935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:55.541888 20935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:55.542881 20935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "cb193c8cfc3f4f3e9501173e23f9b77b"
format_stamp: "Formatted at 2026-09-24 07:14:55 on dist-test-slave-7q7m"
server_key: "de17d309c4c9e6e02cd3231a2c6c71e1"
server_key_iv: "9aabe0d78ea5205b298259230c1c216c"
server_key_version: "encryptionkey@0"
I20260924 07:14:55.543206 20935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "cb193c8cfc3f4f3e9501173e23f9b77b"
format_stamp: "Formatted at 2026-09-24 07:14:55 on dist-test-slave-7q7m"
server_key: "de17d309c4c9e6e02cd3231a2c6c71e1"
server_key_iv: "9aabe0d78ea5205b298259230c1c216c"
server_key_version: "encryptionkey@0"
I20260924 07:14:55.544473 20935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:14:55.545192 20950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:55.545351 20935 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:55.545430 20935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "cb193c8cfc3f4f3e9501173e23f9b77b"
format_stamp: "Formatted at 2026-09-24 07:14:55 on dist-test-slave-7q7m"
server_key: "de17d309c4c9e6e02cd3231a2c6c71e1"
server_key_iv: "9aabe0d78ea5205b298259230c1c216c"
server_key_version: "encryptionkey@0"
I20260924 07:14:55.545488 20935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:55.559696 20935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:55.561458 20935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:55.561596 20935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:55.565215 20935 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:36453
I20260924 07:14:55.565258 21002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:36453 every 8 connection(s)
I20260924 07:14:55.565572 20935 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:14:55.566509 21003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:55.568885 21003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b: Bootstrap starting.
I20260924 07:14:55.569460 21003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:55.569723 21003 log.cc:824] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:55.570387 21003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b: No bootstrap required, opened a new log
I20260924 07:14:55.572135 21003 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb193c8cfc3f4f3e9501173e23f9b77b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36453 } }
I20260924 07:14:55.572314 21003 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:55.572350 21003 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb193c8cfc3f4f3e9501173e23f9b77b, State: Initialized, Role: FOLLOWER
I20260924 07:14:55.572502 21003 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb193c8cfc3f4f3e9501173e23f9b77b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36453 } }
I20260924 07:14:55.572566 21003 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:14:55.572602 21003 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:14:55.572654 21003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:55.573457 21003 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb193c8cfc3f4f3e9501173e23f9b77b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36453 } }
I20260924 07:14:55.573553 21003 leader_election.cc:302] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb193c8cfc3f4f3e9501173e23f9b77b; no voters:
I20260924 07:14:55.573724 21003 leader_election.cc:288] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:14:55.573750 21006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:55.573896 21006 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [term 1 LEADER]: Becoming Leader. State: Replica: cb193c8cfc3f4f3e9501173e23f9b77b, State: Running, Role: LEADER
I20260924 07:14:55.573968 21003 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:14:55.574005 21006 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb193c8cfc3f4f3e9501173e23f9b77b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36453 } }
I20260924 07:14:55.574361 21008 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb193c8cfc3f4f3e9501173e23f9b77b. Latest consensus state: current_term: 1 leader_uuid: "cb193c8cfc3f4f3e9501173e23f9b77b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb193c8cfc3f4f3e9501173e23f9b77b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36453 } } }
I20260924 07:14:55.574534 21008 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:55.574622 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 20935
I20260924 07:14:55.574720 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:14:55.574930 10763 external_mini_cluster.cc:1428] Setting key f43df923eee3ccca06f9093006465bcb
I20260924 07:14:55.574937 21015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:14:55.574352 21007 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb193c8cfc3f4f3e9501173e23f9b77b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb193c8cfc3f4f3e9501173e23f9b77b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36453 } } }
I20260924 07:14:55.575433 21015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:14:55.575250 21007 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b [sys.catalog]: This master's current role is: LEADER
I20260924 07:14:55.577183 21015 catalog_manager.cc:1347] Generated new cluster ID: 3d4608cb21564cb08849dab3067c6e00
I20260924 07:14:55.577227 21015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:14:55 dist-test-slave-7q7m krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:55.587263 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:55.575420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46424 (local address 127.10.130.254:36453)
0924 07:14:55.575570 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:55.575588 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:55.575762 (+ 174us) server_negotiation.cc:407] Connection header received
0924 07:14:55.575915 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:55.575923 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:55.575999 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:55.576126 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:55.577293 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:55.578669 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:55.579748 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:55.580396 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:55.580502 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:55.583314 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:55.583318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:55.583318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:55.583329 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:55.585225 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:55.585640 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:55.585644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:55.585644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:55.585765 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:55.585970 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:55.585970 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:55.585971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:55.586275 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:55.586405 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:55.586900 (+ 495us) server_negotiation.cc:299] Negotiation successful
0924 07:14:55.587038 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":78,"threads_started":1}
I20260924 07:14:55.593158 21015 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:14:55.593644 21015 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:14:55.600718 21015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b: Generated new TSK 0
I20260924 07:14:55.600901 21015 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:14:55.628813 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:36453
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:55.723188 21029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:55.723376 21029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:55.723398 21029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:55.724541 21029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:55.724579 21029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:55.724606 21029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:14:55.726027 21029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:36453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:55.726230 21029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:55.726429 21029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:55.730902 21035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:55.730978 21034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:55.730911 21037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:55.731012 21029 server_base.cc:1034] running on GCE node
I20260924 07:14:55.731315 21029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:55.731531 21029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:55.732661 21029 hybrid_clock.cc:648] HybridClock initialized: now 1790234095732645 us; error 27 us; skew 500 ppm
Sep 24 07:14:55 dist-test-slave-7q7m krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234095, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:55.734974 21029 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:14:55.735358 21029 webserver.cc:460] Webserver started at http://127.10.130.193:34887/ using document root <none> and password file <none>
I20260924 07:14:55.735540 21029 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:55.735594 21029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:55.735714 21029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:55.736735 21029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "5ec8a0b6e13f442fb3bbe657a2c55860"
format_stamp: "Formatted at 2026-09-24 07:14:55 on dist-test-slave-7q7m"
server_key: "49dfeeb7488c614adbd86ba4eeb55797"
server_key_iv: "28a6058bd0694c46034fda285f302f8c"
server_key_version: "encryptionkey@0"
I20260924 07:14:55.737043 21029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "5ec8a0b6e13f442fb3bbe657a2c55860"
format_stamp: "Formatted at 2026-09-24 07:14:55 on dist-test-slave-7q7m"
server_key: "49dfeeb7488c614adbd86ba4eeb55797"
server_key_iv: "28a6058bd0694c46034fda285f302f8c"
server_key_version: "encryptionkey@0"
I20260924 07:14:55.738176 21029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:14:55.738755 21044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:55.738912 21029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:55.738989 21029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "5ec8a0b6e13f442fb3bbe657a2c55860"
format_stamp: "Formatted at 2026-09-24 07:14:55 on dist-test-slave-7q7m"
server_key: "49dfeeb7488c614adbd86ba4eeb55797"
server_key_iv: "28a6058bd0694c46034fda285f302f8c"
server_key_version: "encryptionkey@0"
I20260924 07:14:55.739046 21029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:55.748373 21029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:55.750165 21029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:55.750293 21029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:55.750479 21029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:55.750756 21029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:55.750787 21029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:55.750814 21029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:55.750839 21029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:55.756117 21029 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:37313
I20260924 07:14:55.756174 21157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:37313 every 8 connection(s)
I20260924 07:14:55.756449 21029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
Sep 24 07:14:55 dist-test-slave-7q7m krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234095, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:55.763535 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 21029
I20260924 07:14:55.763638 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:14:55.763841 10763 external_mini_cluster.cc:1428] Setting key 63f5c49d62a64b60f1f2418ec49f7dbd
I20260924 07:14:55.765380 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:55.757140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:53405 (local address 127.10.130.254:36453)
0924 07:14:55.757196 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:55.757197 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:55.757401 (+ 204us) server_negotiation.cc:407] Connection header received
0924 07:14:55.757879 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:55.757880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:55.757890 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:55.757942 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:55.758824 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:55.759320 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:55.760431 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:55.760633 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:55.760685 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:55.762659 (+ 1974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:55.762660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:55.762661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:55.762667 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:55.763947 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:55.764393 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:55.764395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:55.764395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:55.764467 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:55.764767 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:55.764767 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:55.764768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:55.764875 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:55.764950 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:55.765242 (+ 292us) server_negotiation.cc:299] Negotiation successful
0924 07:14:55.765315 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 07:14:55.765604 21160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:55.757118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:36453 (local address 127.10.130.193:53405)
0924 07:14:55.757281 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:55.757322 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:55.757803 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:55.758009 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:55.758013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:55.758566 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:55.758753 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:55.758758 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:55.759403 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:55.759404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:55.760362 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:55.760365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:55.760691 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:55.760692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:55.760789 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:55.761248 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:55.761256 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:55.762572 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:55.764062 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:55.764070 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:55.764072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:55.764313 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:55.764543 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:55.764544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:55.764545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:55.764702 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:55.764952 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:55.764957 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:55.765116 (+ 159us) client_negotiation.cc:769] Sending connection context
0924 07:14:55.765221 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 07:14:55.765379 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260924 07:14:55.766074 21158 heartbeater.cc:347] Connected to a master server at 127.10.130.254:36453
I20260924 07:14:55.766176 21158 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:55.766355 21158 heartbeater.cc:511] Master 127.10.130.254:36453 requested a full tablet report, sending...
I20260924 07:14:55.766868 20967 ts_manager.cc:194] Registered new tserver with Master: 5ec8a0b6e13f442fb3bbe657a2c55860 (127.10.130.193:37313)
I20260924 07:14:55.767488 20967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:53405
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:14:55.786376 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:36453
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:55.862751 21163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:55.862957 21163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:55.862987 21163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:55.864092 21163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:55.864128 21163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:55.864161 21163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:14:55.865507 21163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:36453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:55.865717 21163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:55.865921 21163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:55.870497 21169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:55.870529 21163 server_base.cc:1034] running on GCE node
W20260924 07:14:55.870486 21168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:55.870471 21171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:55.870838 21163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:55.871078 21163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:55.872207 21163 hybrid_clock.cc:648] HybridClock initialized: now 1790234095872192 us; error 27 us; skew 500 ppm
Sep 24 07:14:55 dist-test-slave-7q7m krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234095, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:55.874413 21163 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:14:55.874805 21163 webserver.cc:460] Webserver started at http://127.10.130.194:40453/ using document root <none> and password file <none>
I20260924 07:14:55.875021 21163 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:55.875118 21163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:55.875244 21163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:55.876236 21163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "5094718b80c848ba8b6d821ae486f973"
format_stamp: "Formatted at 2026-09-24 07:14:55 on dist-test-slave-7q7m"
server_key: "f938ce402e2ba67f720d6485d37abb11"
server_key_iv: "8aeff9eef49ca287918ea8b5521ce60c"
server_key_version: "encryptionkey@0"
I20260924 07:14:55.876511 21163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "5094718b80c848ba8b6d821ae486f973"
format_stamp: "Formatted at 2026-09-24 07:14:55 on dist-test-slave-7q7m"
server_key: "f938ce402e2ba67f720d6485d37abb11"
server_key_iv: "8aeff9eef49ca287918ea8b5521ce60c"
server_key_version: "encryptionkey@0"
I20260924 07:14:55.877589 21163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:14:55.878157 21178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:55.878293 21163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:14:55.878363 21163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "5094718b80c848ba8b6d821ae486f973"
format_stamp: "Formatted at 2026-09-24 07:14:55 on dist-test-slave-7q7m"
server_key: "f938ce402e2ba67f720d6485d37abb11"
server_key_iv: "8aeff9eef49ca287918ea8b5521ce60c"
server_key_version: "encryptionkey@0"
I20260924 07:14:55.878424 21163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:55.899596 21163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:55.901510 21163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:55.901634 21163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:55.901831 21163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:55.902098 21163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:55.902129 21163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:55.902149 21163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:55.902172 21163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:55.907423 21163 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:43557
I20260924 07:14:55.907485 21291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:43557 every 8 connection(s)
I20260924 07:14:55.907758 21163 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:14:55.910147 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 21163
I20260924 07:14:55.910233 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:14:55.910405 10763 external_mini_cluster.cc:1428] Setting key d312e46a04018c5558274eaff950913b
Sep 24 07:14:55 dist-test-slave-7q7m krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234095, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:55.916643 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:55.908407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:58905 (local address 127.10.130.254:36453)
0924 07:14:55.908457 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:55.908459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:55.908624 (+ 165us) server_negotiation.cc:407] Connection header received
0924 07:14:55.909114 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:55.909115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:55.909128 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:55.909186 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:55.910083 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:55.910721 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:55.912014 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:55.912209 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:55.912254 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:55.914246 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:55.914248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:55.914248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:55.914255 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:55.915425 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:55.915775 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:55.915776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:55.915776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:55.915826 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:55.916063 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:55.916064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:55.916064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:55.916167 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:55.916222 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:55.916472 (+ 250us) server_negotiation.cc:299] Negotiation successful
0924 07:14:55.916557 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:14:55.916780 21294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:55.908375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:36453 (local address 127.10.130.194:58905)
0924 07:14:55.908511 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:55.908548 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:55.909020 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:55.909257 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:55.909261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:55.909789 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:55.909983 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:55.909989 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:55.910794 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:55.910795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:55.911912 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:55.911915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:55.912274 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:55.912275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:55.912419 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:55.912884 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:55.912891 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:55.914164 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:55.915502 (+ 1338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:55.915508 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:55.915511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:55.915713 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:55.915878 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:55.915879 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:55.915879 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:55.916013 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:55.916218 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:55.916224 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:55.916350 (+ 126us) client_negotiation.cc:769] Sending connection context
0924 07:14:55.916441 (+ 91us) client_negotiation.cc:240] Negotiation successful
0924 07:14:55.916580 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260924 07:14:55.917194 21292 heartbeater.cc:347] Connected to a master server at 127.10.130.254:36453
I20260924 07:14:55.917275 21292 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:55.917433 21292 heartbeater.cc:511] Master 127.10.130.254:36453 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
I20260924 07:14:55.917793 20967 ts_manager.cc:194] Registered new tserver with Master: 5094718b80c848ba8b6d821ae486f973 (127.10.130.194:43557)
I20260924 07:14:55.918134 20967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:58905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:14:55.931519 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:36453
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:14:56.008747 21297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:14:56.008935 21297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:14:56.008956 21297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:14:56.010010 21297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:14:56.010044 21297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:14:56.010068 21297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:14:56.011880 21297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:36453
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:14:56.012112 21297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:14:56.012293 21297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:14:56.016992 21302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:14:56.016999 21305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:56.017066 21297 server_base.cc:1034] running on GCE node
W20260924 07:14:56.016999 21303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:14:56.017335 21297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:14:56.017555 21297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:14:56.018684 21297 hybrid_clock.cc:648] HybridClock initialized: now 1790234096018674 us; error 29 us; skew 500 ppm
Sep 24 07:14:56 dist-test-slave-7q7m krb5kdc[20922](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234096, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:14:56.020973 21297 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:14:56.021371 21297 webserver.cc:460] Webserver started at http://127.10.130.195:39415/ using document root <none> and password file <none>
I20260924 07:14:56.021569 21297 fs_manager.cc:359] Metadata directory not provided
I20260924 07:14:56.021620 21297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:14:56.021735 21297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:14:56.022771 21297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "b6031b9d3b9240a8b2680bd0f114f4af"
format_stamp: "Formatted at 2026-09-24 07:14:56 on dist-test-slave-7q7m"
server_key: "9dbaf45e5f08d6cd2051de12ac75ddb3"
server_key_iv: "6ff8893a349474063839231cde654e7f"
server_key_version: "encryptionkey@0"
I20260924 07:14:56.023080 21297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "b6031b9d3b9240a8b2680bd0f114f4af"
format_stamp: "Formatted at 2026-09-24 07:14:56 on dist-test-slave-7q7m"
server_key: "9dbaf45e5f08d6cd2051de12ac75ddb3"
server_key_iv: "6ff8893a349474063839231cde654e7f"
server_key_version: "encryptionkey@0"
I20260924 07:14:56.024273 21297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:14:56.024887 21312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:56.025032 21297 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:14:56.025110 21297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "b6031b9d3b9240a8b2680bd0f114f4af"
format_stamp: "Formatted at 2026-09-24 07:14:56 on dist-test-slave-7q7m"
server_key: "9dbaf45e5f08d6cd2051de12ac75ddb3"
server_key_iv: "6ff8893a349474063839231cde654e7f"
server_key_version: "encryptionkey@0"
I20260924 07:14:56.025168 21297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:14:56.037855 21297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:14:56.039443 21297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:14:56.039565 21297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:14:56.039755 21297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:14:56.040040 21297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:14:56.040079 21297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:56.040107 21297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:14:56.040130 21297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:14:56.045563 21297 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:39701
I20260924 07:14:56.045598 21425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:39701 every 8 connection(s)
I20260924 07:14:56.045881 21297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
Sep 24 07:14:56 dist-test-slave-7q7m krb5kdc[20922](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234096, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:14:56.053932 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.046479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:43897 (local address 127.10.130.254:36453)
0924 07:14:56.046536 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:56.046537 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:56.046721 (+ 184us) server_negotiation.cc:407] Connection header received
0924 07:14:56.047216 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:56.047218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:56.047228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:56.047289 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:56.048257 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.048788 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.049704 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.049883 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.049928 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.051722 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:56.051723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:56.051724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:56.051729 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:56.052949 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:56.053266 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:56.053267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:56.053267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:56.053310 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:56.053487 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:56.053488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:56.053488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:56.053577 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:56.053620 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:56.053820 (+ 200us) server_negotiation.cc:299] Negotiation successful
0924 07:14:56.053870 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 07:14:56.054090 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.046448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:36453 (local address 127.10.130.195:43897)
0924 07:14:56.046601 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:56.046640 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:56.047117 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:56.047350 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:56.047353 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:56.047941 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:14:56.048178 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:56.048184 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.048856 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.048857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:56.049639 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:56.049642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.049937 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.049938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:56.050034 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.050408 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:14:56.050415 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:14:56.051647 (+ 1232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:14:56.053019 (+ 1372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:56.053022 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:56.053023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:56.053217 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:56.053353 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:14:56.053354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:14:56.053354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:14:56.053445 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:14:56.053618 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:14:56.053621 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:14:56.053723 (+ 102us) client_negotiation.cc:769] Sending connection context
0924 07:14:56.053795 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 07:14:56.053901 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260924 07:14:56.054498 21426 heartbeater.cc:347] Connected to a master server at 127.10.130.254:36453
I20260924 07:14:56.054574 21426 heartbeater.cc:464] Registering TS with master...
I20260924 07:14:56.054726 21426 heartbeater.cc:511] Master 127.10.130.254:36453 requested a full tablet report, sending...
I20260924 07:14:56.055037 20967 ts_manager.cc:194] Registered new tserver with Master: b6031b9d3b9240a8b2680bd0f114f4af (127.10.130.195:39701)
I20260924 07:14:56.055383 20967 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:43897
I20260924 07:14:56.055392 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 21297
I20260924 07:14:56.055478 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:14:56.055634 10763 external_mini_cluster.cc:1428] Setting key b790de747522fce70a7bf438865ff799
I20260924 07:14:56.056403 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:14:56.062422 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.057549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46438 (local address 127.10.130.254:36453)
0924 07:14:56.057605 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:56.057607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:56.057674 (+ 67us) server_negotiation.cc:407] Connection header received
0924 07:14:56.057719 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:56.057720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:56.057727 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:56.057778 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:14:56.058240 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.058686 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.059336 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.059500 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.059539 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.060573 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:14:56.060573 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:14:56.060574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:14:56.060578 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:14:56.061606 (+ 1028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:56.061875 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:56.061876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:56.061876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:56.061919 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:14:56.062032 (+ 113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:14:56.062033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:14:56.062033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:14:56.062150 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:14:56.062215 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:56.062291 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 07:14:56.062341 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 07:14:56.063678 20967 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:14:56.065196 20967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:14:56.071123 21436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.067169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43966 (local address 127.10.130.194:43557)
0924 07:14:56.067336 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:56.067341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:56.067558 (+ 217us) server_negotiation.cc:407] Connection header received
0924 07:14:56.067602 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:56.067605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:56.067621 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:56.067732 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:56.067888 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.069015 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.070292 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.070751 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.070818 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.070873 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:56.070996 (+ 123us) server_negotiation.cc:299] Negotiation successful
0924 07:14:56.071032 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":109,"threads_started":1}
I20260924 07:14:56.071116 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.067072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:43557 (local address 127.0.0.1:43966)
0924 07:14:56.067378 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:56.067416 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:56.067484 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:56.067741 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:56.067744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:56.067746 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:56.067804 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:56.067806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.069110 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.069111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:56.070139 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:56.070141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.070826 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.070827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:56.070902 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.070914 (+ 12us) client_negotiation.cc:769] Sending connection context
0924 07:14:56.070978 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 07:14:56.071030 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":53,"threads_started":1}
I20260924 07:14:56.071229 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.067072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:39701 (local address 127.0.0.1:55348)
0924 07:14:56.067386 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:56.067418 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:56.067482 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:56.068137 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:56.068138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:56.068140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:56.068181 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:56.068182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.069315 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.069316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:56.070187 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:56.070190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.071051 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.071051 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:56.071103 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.071106 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:56.071144 (+ 38us) client_negotiation.cc:240] Negotiation successful
0924 07:14:56.071176 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":2,"thread_start_us":56,"threads_started":1}
I20260924 07:14:56.071290 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.067449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45262 (local address 127.10.130.193:37313)
0924 07:14:56.067660 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:56.067664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:56.067673 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:14:56.067706 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:56.067708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:56.067722 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:56.067790 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:56.067922 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.068727 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.070044 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.070757 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.070824 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.070873 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:56.071173 (+ 300us) server_negotiation.cc:299] Negotiation successful
0924 07:14:56.071216 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":80,"threads_started":1}
I20260924 07:14:56.071137 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.067243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:37313 (local address 127.0.0.1:45262)
0924 07:14:56.067436 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:56.067445 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:56.067495 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:56.067786 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:56.067787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:56.067788 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:56.067815 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:56.067816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.068813 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.068814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:56.069976 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:56.069978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.070906 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.070907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:56.070965 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.070969 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:56.071010 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 07:14:56.071051 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":60,"threads_started":1}
I20260924 07:14:56.071492 21438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.067224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55348 (local address 127.10.130.195:39701)
0924 07:14:56.067903 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:56.067907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:56.067915 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:56.067948 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:56.067950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:56.067964 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:56.068051 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:56.068256 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.069236 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.070268 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.070905 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.070961 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.071022 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:56.071377 (+ 355us) server_negotiation.cc:299] Negotiation successful
0924 07:14:56.071421 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":85,"threads_started":1}
I20260924 07:14:56.071875 21092 tablet_service.cc:1467] Processing CreateTablet for tablet 3863cf73ccf44a0bb68c1070cd3a42de (DEFAULT_TABLE table=test-table [id=ece1abaa628245048a54a7beb62ed12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:56.072229 21092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3863cf73ccf44a0bb68c1070cd3a42de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:56.072259 21226 tablet_service.cc:1467] Processing CreateTablet for tablet 3863cf73ccf44a0bb68c1070cd3a42de (DEFAULT_TABLE table=test-table [id=ece1abaa628245048a54a7beb62ed12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:56.072566 21226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3863cf73ccf44a0bb68c1070cd3a42de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:56.074925 21440 tablet_bootstrap.cc:492] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860: Bootstrap starting.
I20260924 07:14:56.075383 21441 tablet_bootstrap.cc:492] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973: Bootstrap starting.
I20260924 07:14:56.075387 21360 tablet_service.cc:1467] Processing CreateTablet for tablet 3863cf73ccf44a0bb68c1070cd3a42de (DEFAULT_TABLE table=test-table [id=ece1abaa628245048a54a7beb62ed12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:14:56.075646 21360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3863cf73ccf44a0bb68c1070cd3a42de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:14:56.075733 21440 tablet_bootstrap.cc:654] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:56.076038 21441 tablet_bootstrap.cc:654] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:56.076341 21440 log.cc:824] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:56.076613 21441 log.cc:824] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:56.077083 21440 tablet_bootstrap.cc:492] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860: No bootstrap required, opened a new log
I20260924 07:14:56.077154 21440 ts_tablet_manager.cc:1397] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:14:56.077227 21441 tablet_bootstrap.cc:492] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973: No bootstrap required, opened a new log
I20260924 07:14:56.077283 21441 ts_tablet_manager.cc:1397] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 07:14:56.078058 21444 tablet_bootstrap.cc:492] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af: Bootstrap starting.
I20260924 07:14:56.078640 21444 tablet_bootstrap.cc:654] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af: Neither blocks nor log segments found. Creating new log.
I20260924 07:14:56.078990 21444 log.cc:824] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af: Log is configured to *not* fsync() on all Append() calls
I20260924 07:14:56.079027 21441 raft_consensus.cc:348] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6031b9d3b9240a8b2680bd0f114f4af" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39701 } } peers { permanent_uuid: "5094718b80c848ba8b6d821ae486f973" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43557 } } peers { permanent_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 37313 } }
I20260924 07:14:56.079027 21440 raft_consensus.cc:348] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6031b9d3b9240a8b2680bd0f114f4af" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39701 } } peers { permanent_uuid: "5094718b80c848ba8b6d821ae486f973" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43557 } } peers { permanent_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 37313 } }
I20260924 07:14:56.079180 21440 raft_consensus.cc:374] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:56.079180 21441 raft_consensus.cc:374] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:56.079212 21440 raft_consensus.cc:729] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec8a0b6e13f442fb3bbe657a2c55860, State: Initialized, Role: FOLLOWER
I20260924 07:14:56.079212 21441 raft_consensus.cc:729] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5094718b80c848ba8b6d821ae486f973, State: Initialized, Role: FOLLOWER
I20260924 07:14:56.079316 21440 consensus_queue.cc:260] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6031b9d3b9240a8b2680bd0f114f4af" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39701 } } peers { permanent_uuid: "5094718b80c848ba8b6d821ae486f973" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43557 } } peers { permanent_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 37313 } }
I20260924 07:14:56.079316 21441 consensus_queue.cc:260] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6031b9d3b9240a8b2680bd0f114f4af" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39701 } } peers { permanent_uuid: "5094718b80c848ba8b6d821ae486f973" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43557 } } peers { permanent_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 37313 } }
I20260924 07:14:56.079572 21441 ts_tablet_manager.cc:1428] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 07:14:56.079572 21440 ts_tablet_manager.cc:1428] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 07:14:56.079612 21292 heartbeater.cc:503] Master 127.10.130.254:36453 was elected leader, sending a full tablet report...
I20260924 07:14:56.079825 21158 heartbeater.cc:503] Master 127.10.130.254:36453 was elected leader, sending a full tablet report...
I20260924 07:14:56.079571 21444 tablet_bootstrap.cc:492] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af: No bootstrap required, opened a new log
I20260924 07:14:56.080083 21444 ts_tablet_manager.cc:1397] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:14:56.081647 21444 raft_consensus.cc:348] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6031b9d3b9240a8b2680bd0f114f4af" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39701 } } peers { permanent_uuid: "5094718b80c848ba8b6d821ae486f973" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43557 } } peers { permanent_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 37313 } }
I20260924 07:14:56.081779 21444 raft_consensus.cc:374] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:14:56.081801 21444 raft_consensus.cc:729] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6031b9d3b9240a8b2680bd0f114f4af, State: Initialized, Role: FOLLOWER
I20260924 07:14:56.081900 21444 consensus_queue.cc:260] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6031b9d3b9240a8b2680bd0f114f4af" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39701 } } peers { permanent_uuid: "5094718b80c848ba8b6d821ae486f973" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43557 } } peers { permanent_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 37313 } }
I20260924 07:14:56.082137 21444 ts_tablet_manager.cc:1428] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:14:56.082170 21426 heartbeater.cc:503] Master 127.10.130.254:36453 was elected leader, sending a full tablet report...
W20260924 07:14:56.158288 21293 tablet.cc:2369] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:14:56.207881 21447 raft_consensus.cc:482] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:14:56.208022 21447 raft_consensus.cc:504] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6031b9d3b9240a8b2680bd0f114f4af" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39701 } } peers { permanent_uuid: "5094718b80c848ba8b6d821ae486f973" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43557 } } peers { permanent_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 37313 } }
I20260924 07:14:56.208397 21447 leader_election.cc:288] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6031b9d3b9240a8b2680bd0f114f4af (127.10.130.195:39701), 5094718b80c848ba8b6d821ae486f973 (127.10.130.194:43557)
I20260924 07:14:56.211355 21438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.208689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:38399 (local address 127.10.130.195:39701)
0924 07:14:56.208742 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:56.208743 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:56.208983 (+ 240us) server_negotiation.cc:407] Connection header received
0924 07:14:56.209005 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:56.209006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:56.209012 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:56.209052 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:56.209269 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.209819 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.210695 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.211061 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.211121 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.211130 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:56.211257 (+ 127us) server_negotiation.cc:299] Negotiation successful
0924 07:14:56.211289 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 07:14:56.211364 21436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.208679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:56453 (local address 127.10.130.194:43557)
0924 07:14:56.208726 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:56.208728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:56.208738 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:14:56.208769 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:56.208770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:56.208781 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:56.208841 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:14:56.208989 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.209665 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.210632 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.211031 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.211101 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.211111 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:56.211255 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 07:14:56.211291 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 07:14:56.211404 21449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.208587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:39701 (local address 127.10.130.193:38399)
0924 07:14:56.208789 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:56.208796 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:56.208837 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:56.209068 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:56.209069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:56.209071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:56.209200 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:56.209202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.209892 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.209893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:56.210615 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:56.210617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.211134 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.211135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:56.211199 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.211202 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:14:56.211246 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 07:14:56.211285 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":124,"threads_started":1}
I20260924 07:14:56.211407 21160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.208578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:43557 (local address 127.10.130.193:56453)
0924 07:14:56.208647 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:56.208658 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:14:56.208701 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:14:56.208858 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:14:56.208859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:14:56.208861 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:14:56.208926 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:56.208928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.209748 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.209749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:56.210539 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:14:56.210542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.211110 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.211111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:14:56.211188 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.211192 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:14:56.211244 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 07:14:56.211285 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":23}
I20260924 07:14:56.211791 21246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3863cf73ccf44a0bb68c1070cd3a42de" candidate_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5094718b80c848ba8b6d821ae486f973" is_pre_election: true
I20260924 07:14:56.211865 21380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3863cf73ccf44a0bb68c1070cd3a42de" candidate_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6031b9d3b9240a8b2680bd0f114f4af" is_pre_election: true
I20260924 07:14:56.211927 21246 raft_consensus.cc:2454] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ec8a0b6e13f442fb3bbe657a2c55860 in term 0.
I20260924 07:14:56.212040 21380 raft_consensus.cc:2454] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ec8a0b6e13f442fb3bbe657a2c55860 in term 0.
I20260924 07:14:56.212203 21046 leader_election.cc:302] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5094718b80c848ba8b6d821ae486f973, 5ec8a0b6e13f442fb3bbe657a2c55860; no voters:
I20260924 07:14:56.212340 21447 raft_consensus.cc:2781] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:14:56.212410 21447 raft_consensus.cc:482] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:14:56.212445 21447 raft_consensus.cc:3037] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:56.213151 21447 raft_consensus.cc:504] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6031b9d3b9240a8b2680bd0f114f4af" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39701 } } peers { permanent_uuid: "5094718b80c848ba8b6d821ae486f973" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43557 } } peers { permanent_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 37313 } }
I20260924 07:14:56.213270 21447 leader_election.cc:288] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [CANDIDATE]: Term 1 election: Requested vote from peers b6031b9d3b9240a8b2680bd0f114f4af (127.10.130.195:39701), 5094718b80c848ba8b6d821ae486f973 (127.10.130.194:43557)
I20260924 07:14:56.213421 21380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3863cf73ccf44a0bb68c1070cd3a42de" candidate_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6031b9d3b9240a8b2680bd0f114f4af"
I20260924 07:14:56.213433 21246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3863cf73ccf44a0bb68c1070cd3a42de" candidate_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5094718b80c848ba8b6d821ae486f973"
I20260924 07:14:56.213495 21246 raft_consensus.cc:3037] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:56.213500 21380 raft_consensus.cc:3037] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:14:56.214182 21246 raft_consensus.cc:2454] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec8a0b6e13f442fb3bbe657a2c55860 in term 1.
I20260924 07:14:56.214238 21380 raft_consensus.cc:2454] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec8a0b6e13f442fb3bbe657a2c55860 in term 1.
I20260924 07:14:56.214380 21047 leader_election.cc:302] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ec8a0b6e13f442fb3bbe657a2c55860, b6031b9d3b9240a8b2680bd0f114f4af; no voters:
I20260924 07:14:56.214475 21447 raft_consensus.cc:2781] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:14:56.214630 21447 raft_consensus.cc:686] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [term 1 LEADER]: Becoming Leader. State: Replica: 5ec8a0b6e13f442fb3bbe657a2c55860, State: Running, Role: LEADER
I20260924 07:14:56.214720 21447 consensus_queue.cc:237] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6031b9d3b9240a8b2680bd0f114f4af" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39701 } } peers { permanent_uuid: "5094718b80c848ba8b6d821ae486f973" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43557 } } peers { permanent_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 37313 } }
I20260924 07:14:56.215476 20967 catalog_manager.cc:5495] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ec8a0b6e13f442fb3bbe657a2c55860 (127.10.130.193). New cstate: current_term: 1 leader_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6031b9d3b9240a8b2680bd0f114f4af" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5094718b80c848ba8b6d821ae486f973" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43557 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec8a0b6e13f442fb3bbe657a2c55860" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 37313 } health_report { overall_health: HEALTHY } } }
I20260924 07:14:56.249114 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.246974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.10.130.193:37313)
0924 07:14:56.247022 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 07:14:56.247023 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:14:56.247031 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:14:56.247066 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:14:56.247067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:14:56.247084 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:14:56.247146 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:14:56.247243 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.247732 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.248490 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:14:56.248690 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:14:56.248738 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:14:56.248854 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:14:56.248957 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:14:56.249013 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 07:14:56.249026 (+ 13us) server_negotiation.cc:299] Negotiation successful
0924 07:14:56.249042 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 07:14:56.250753 21246 raft_consensus.cc:1261] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973 [term 1 FOLLOWER]: Refusing update from remote peer 5ec8a0b6e13f442fb3bbe657a2c55860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:56.250759 21380 raft_consensus.cc:1261] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af [term 1 FOLLOWER]: Refusing update from remote peer 5ec8a0b6e13f442fb3bbe657a2c55860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:14:56.251107 21450 consensus_queue.cc:1035] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6031b9d3b9240a8b2680bd0f114f4af" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:56.251237 21447 consensus_queue.cc:1035] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5094718b80c848ba8b6d821ae486f973" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:14:56.253564 21455 mvcc.cc:205] Tried to move back new op lower bound from 7332798858240569344 to 7332798858095964160. Current Snapshot: MvccSnapshot[applied={T|T < 7332798858240569344}]
I20260924 07:14:56.254033 21454 mvcc.cc:205] Tried to move back new op lower bound from 7332798858240569344 to 7332798858095964160. Current Snapshot: MvccSnapshot[applied={T|T < 7332798858240569344}]
I20260924 07:14:56.256346 20967 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:56.256433 20967 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:14:56.257122 20967 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb193c8cfc3f4f3e9501173e23f9b77b: Sending DeleteTablet for 3 replicas of tablet 3863cf73ccf44a0bb68c1070cd3a42de
I20260924 07:14:56.257457 21226 tablet_service.cc:1514] Processing DeleteTablet for tablet 3863cf73ccf44a0bb68c1070cd3a42de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:56 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:43966
I20260924 07:14:56.257462 21092 tablet_service.cc:1514] Processing DeleteTablet for tablet 3863cf73ccf44a0bb68c1070cd3a42de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:56 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:45262
I20260924 07:14:56.257462 21360 tablet_service.cc:1514] Processing DeleteTablet for tablet 3863cf73ccf44a0bb68c1070cd3a42de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:14:56 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:55348
I20260924 07:14:56.257710 21465 tablet_replica.cc:330] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860: stopping tablet replica
I20260924 07:14:56.257824 21464 tablet_replica.cc:330] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973: stopping tablet replica
I20260924 07:14:56.257843 21465 raft_consensus.cc:2229] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:14:56.257933 21464 raft_consensus.cc:2229] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:56.257956 21465 raft_consensus.cc:2258] T 3863cf73ccf44a0bb68c1070cd3a42de P 5ec8a0b6e13f442fb3bbe657a2c55860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:56.257987 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 21029
I20260924 07:14:56.258039 21464 raft_consensus.cc:2258] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:56.258405 21464 ts_tablet_manager.cc:1905] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:56.259640 21463 tablet_replica.cc:330] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af: stopping tablet replica
I20260924 07:14:56.259711 21463 raft_consensus.cc:2229] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:14:56.259775 21463 raft_consensus.cc:2258] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:14:56.259935 21464 ts_tablet_manager.cc:1918] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:56.260010 21464 log.cc:1196] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/3863cf73ccf44a0bb68c1070cd3a42de
I20260924 07:14:56.260236 21463 ts_tablet_manager.cc:1905] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:14:56.260253 21464 ts_tablet_manager.cc:1939] T 3863cf73ccf44a0bb68c1070cd3a42de P 5094718b80c848ba8b6d821ae486f973: Deleting consensus metadata
I20260924 07:14:56.261042 20952 catalog_manager.cc:4841] TS 5094718b80c848ba8b6d821ae486f973 (127.10.130.194:43557): tablet 3863cf73ccf44a0bb68c1070cd3a42de (table test-table [id=ece1abaa628245048a54a7beb62ed12c]) successfully deleted
I20260924 07:14:56.261515 21463 ts_tablet_manager.cc:1918] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:14:56.261569 21463 log.cc:1196] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/3863cf73ccf44a0bb68c1070cd3a42de
I20260924 07:14:56.261747 21463 ts_tablet_manager.cc:1939] T 3863cf73ccf44a0bb68c1070cd3a42de P b6031b9d3b9240a8b2680bd0f114f4af: Deleting consensus metadata
I20260924 07:14:56.262117 20953 catalog_manager.cc:4841] TS b6031b9d3b9240a8b2680bd0f114f4af (127.10.130.195:39701): tablet 3863cf73ccf44a0bb68c1070cd3a42de (table test-table [id=ece1abaa628245048a54a7beb62ed12c]) successfully deleted
W20260924 07:14:56.263604 20952 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:37313 (error 108)
I20260924 07:14:56.264047 21435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:14:56.263930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:37313 (local address 127.0.0.1:45282)
0924 07:14:56.263962 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0924 07:14:56.264015 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:37313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260924 07:14:56.264101 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 21163
W20260924 07:14:56.264761 20952 catalog_manager.cc:4584] TS 5ec8a0b6e13f442fb3bbe657a2c55860 (127.10.130.193:37313): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3863cf73ccf44a0bb68c1070cd3a42de: Network error: Client connection negotiation failed: client connection to 127.10.130.193:37313: connect: Connection refused (error 111)
I20260924 07:14:56.268642 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 21297
I20260924 07:14:56.272533 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 20935
2026-09-24T07:14:56Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1868 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:14:56 dist-test-slave-7q7m krb5kdc[21469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:14:56 dist-test-slave-7q7m krb5kdc[21469](info): set up 2 sockets
Sep 24 07:14:56 dist-test-slave-7q7m krb5kdc[21469](info): commencing operation
krb5kdc: starting...
W20260924 07:14:57.232537 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.942s user 0.002s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:14:57 dist-test-slave-7q7m krb5kdc[21469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234097, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:14:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:14:57Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.130.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.130.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/rangeradmin_127.10.130.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/rangeradmin_127.10.130.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.130.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.130.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/rangerlookup_127.10.130.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/rangerlookup_127.10.130.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.130.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.130.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/HTTP_127.10.130.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/HTTP_127.10.130.212@KRBTEST.COM.keytab.
I20260924 07:14:57.377643 10763 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task9LiIRP/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/postgres -l logfile start
2026-09-24 07:14:59.474 UTC [21499] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 07:14:59.474 UTC [21499] LOG: listening on IPv4 address "127.10.130.212", port 54585
2026-09-24 07:14:59.493 UTC [21499] LOG: listening on Unix socket "/tmp/.s.PGSQL.54585"
2026-09-24 07:14:59.511 UTC [21502] LOG: database system was shut down at 2026-09-24 07:14:58 UTC
2026-09-24 07:14:59.514 UTC [21499] LOG: database system is ready to accept connections
I20260924 07:15:00.364389 10763 mini_postgres.cc:96] Postgres bound to 54585
2026-09-24 07:15:00.369 UTC [21510] FATAL: database "slave" does not exist
127.10.130.212:54585 - accepting connections
I20260924 07:15:00.370414 10763 mini_ranger.cc:161] Starting Ranger...
I20260924 07:15:00.383574 10763 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 07:15:00.744127 10763 mini_ranger.cc:87] Created ranger Postgres database
I20260924 07:15:00.744196 10763 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-admin
2026-09-24 07:15:01,001 [I] DB FLAVOR :POSTGRES
2026-09-24 07:15:01,002 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 07:15:01,002 [I] Checking connection..
2026-09-24 07:15:01,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 07:15:01,235 [I] Checking connection passed.
2026-09-24 07:15:01,235 [I] --------- Verifying version history table ---------
2026-09-24 07:15:01,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 07:15:01,470 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 07:15:01,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 07:15:01,692 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 07:15:01,693 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 07:15:01,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 07:15:01.939 UTC [21574] WARNING: there is no transaction in progress
2026-09-24 07:15:01,944 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 07:15:01,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 07:15:02,183 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 07:15:02,184 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 07:15:02,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 07:15:02,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 07:15:02,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7q7m.c.gcp-upstream.internal','N') ;"
2026-09-24 07:15:02,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 07:15:03,162 [I] Table x_portal_user does not exist in database ranger
2026-09-24 07:15:03,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 07:15:03,390 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 07:15:03,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 07:15:03,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 07:15:03,878 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 07:15:03,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 07:15:04.770 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.786 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.800 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.815 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.831 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.847 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.898 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.904 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.917 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.927 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.936 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.948 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.957 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.963 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.969 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:04.975 UTC [21682] WARNING: there is no transaction in progress
2026-09-24 07:15:05,521 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 07:15:05,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 07:15:05,767 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 07:15:05,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 07:15:06,003 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 07:15:06,004 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 07:15:06,235 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 07:15:06,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 07:15:06,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 07:15:06,733 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7q7m.c.gcp-upstream.internal';"
2026-09-24 07:15:06,973 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 07:15:06,974 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 07:15:07,207 [I] DB_PATCHES have already been applied
I20260924 07:15:07.212684 10763 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-admin:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task9LiIRP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 07:15:07.212759 10763 mini_ranger.cc:193] Using host: 127.10.130.212
I20260924 07:15:07.215622 10763 mini_ranger.cc:227] Ranger admin URL: http://127.10.130.212:51715
Sep 24, 2026 7:15:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 7:15:07 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 7:15:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 7:15:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 7:15:07 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 7:15:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 7:15:08 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 07:15:08.227723 10763 mini_ranger.cc:160] Time spent starting Ranger: real 7.857s user 0.000s sys 0.008s
Sep 24, 2026 7:15:26 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 7:15:26 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 24, 2026 7:15:26 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 7:15:26 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 7:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 7:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 7:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 7:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 7:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 7:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 7:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 7:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 7:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 7:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 7:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 7:15:26 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 7:15:26 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 7:15:27 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 07:15:28.259788 10763 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.130.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.130.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/rangerkms_127.10.130.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/rangerkms_127.10.130.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.130.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.130.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.130.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/HTTP_127.10.130.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.130.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/HTTP_127.10.130.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 07:15:28 dist-test-slave-7q7m krb5kdc[21469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234128, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 07:15:28.336854 10763 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 07:15:28.345192 10763 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 07:15:29.068203 10763 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 07:15:29.068269 10763 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-kms
I20260924 07:15:29.068285 10763 mini_ranger_kms.cc:227] Using postgres at 127.10.130.212:54585
2026-09-24 07:15:29,215 [I] DB FLAVOR :POSTGRES
2026-09-24 07:15:29,215 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 07:15:29,216 [I] Checking connection
2026-09-24 07:15:29,216 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 07:15:29,456 [I] connection success
2026-09-24 07:15:29,456 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 07:15:29,456 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 07:15:29,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 07:15:29,760 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 07:15:29,760 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 07:15:29,760 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 07:15:29,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:54585/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 07:15:29,997 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 07:15:30.387953 10763 mini_ranger_kms.cc:310] Created kms service
I20260924 07:15:30.594122 10763 mini_ranger_kms.cc:326] Created kudu user
I20260924 07:15:30.638768 10763 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 07:15:30 dist-test-slave-7q7m krb5kdc[21469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234128, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:15:31.033545 10763 mini_ranger_kms.cc:384] Added ranger policy
I20260924 07:15:31.033658 10763 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-kms:/tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 07:15:31.033680 10763 mini_ranger_kms.cc:240] Using host: 127.10.130.212
I20260924 07:15:31.037137 10763 mini_ranger_kms.cc:276] Ranger KMS PID: 21862
I20260924 07:15:31.037272 10763 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.130.212:46871
Sep 24, 2026 7:15:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 7:15:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 7:15:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 7:15:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 7:15:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 7:15:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260924 07:15:33.064956 10763 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.728s user 0.000s sys 0.131s
I20260924 07:15:33.065052 10763 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 07:15:33.065129 10763 mini_ranger_kms.cc:401] 127.10.130.212:46871/kms/v1/keys
Sep 24 07:15:33 dist-test-slave-7q7m krb5kdc[21469](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790234133, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.130.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 24 07:15:33 dist-test-slave-7q7m krb5kdc[21469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234133, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.130.212@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
Sep 24 07:15:33 dist-test-slave-7q7m krb5kdc[21469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234133, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.130.212@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
Sep 24 07:15:36 dist-test-slave-7q7m krb5kdc[21469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:15:36.181833 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:40429
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:36743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:40429
--ranger_config_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.130.212:46871/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:15:36.260115 21913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:15:36.260313 21913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:15:36.260334 21913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:15:36.261474 21913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:15:36.261515 21913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:15:36.261528 21913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:15:36.261539 21913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:15:36.261549 21913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:15:36.263063 21913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:36743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.130.212:46871/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:40429
--ranger_config_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:40429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:15:36.263273 21913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:15:36.263473 21913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 07:15:36.268242 21913 server_base.cc:1034] running on GCE node
W20260924 07:15:36.268421 21921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:15:36.268249 21919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:15:36.268421 21918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:15:36.268883 21913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:15:36.269205 21913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:15:36.270377 21913 hybrid_clock.cc:648] HybridClock initialized: now 1790234136270364 us; error 35 us; skew 500 ppm
Sep 24 07:15:36 dist-test-slave-7q7m krb5kdc[21469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234136, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:15:36.276471 21913 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:15:36.276918 21913 webserver.cc:460] Webserver started at http://127.10.130.254:44531/ using document root <none> and password file <none>
I20260924 07:15:36.277124 21913 fs_manager.cc:359] Metadata directory not provided
I20260924 07:15:36.277161 21913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:15:36.277277 21913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 07:15:36 dist-test-slave-7q7m krb5kdc[21469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234136, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:15:36.333344 21913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "fdb031c8331e48159006ae20ce47a684"
format_stamp: "Formatted at 2026-09-24 07:15:36 on dist-test-slave-7q7m"
server_key: "fc0d7d1292f4847ebabcda6bbd77ea81"
server_key_iv: "09f2f9f9fa53ca6e45fb6621aa654edb"
server_key_version: "kuduclusterkey@0"
I20260924 07:15:36.333698 21913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "fdb031c8331e48159006ae20ce47a684"
format_stamp: "Formatted at 2026-09-24 07:15:36 on dist-test-slave-7q7m"
server_key: "fc0d7d1292f4847ebabcda6bbd77ea81"
server_key_iv: "09f2f9f9fa53ca6e45fb6621aa654edb"
server_key_version: "kuduclusterkey@0"
I20260924 07:15:36.334988 21913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 07:15:36.349808 21928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:15:36.350044 21913 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:15:36.350165 21913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "fdb031c8331e48159006ae20ce47a684"
format_stamp: "Formatted at 2026-09-24 07:15:36 on dist-test-slave-7q7m"
server_key: "fc0d7d1292f4847ebabcda6bbd77ea81"
server_key_iv: "09f2f9f9fa53ca6e45fb6621aa654edb"
server_key_version: "kuduclusterkey@0"
I20260924 07:15:36.350286 21913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:15:36.361640 21913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:15:36.363468 21913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:15:36.363617 21913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:15:36.368388 21913 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:40429
I20260924 07:15:36.368422 21980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:40429 every 8 connection(s)
I20260924 07:15:36.368814 21913 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:15:36.369482 21981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:15:36.371608 21981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684: Bootstrap starting.
I20260924 07:15:36.372164 21981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684: Neither blocks nor log segments found. Creating new log.
I20260924 07:15:36.372412 21981 log.cc:824] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684: Log is configured to *not* fsync() on all Append() calls
I20260924 07:15:36.372995 21981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684: No bootstrap required, opened a new log
I20260924 07:15:36.374979 21981 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb031c8331e48159006ae20ce47a684" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40429 } }
I20260924 07:15:36.375116 21981 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:15:36.375147 21981 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdb031c8331e48159006ae20ce47a684, State: Initialized, Role: FOLLOWER
I20260924 07:15:36.375279 21981 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb031c8331e48159006ae20ce47a684" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40429 } }
I20260924 07:15:36.375350 21981 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:15:36.375379 21981 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:15:36.375424 21981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:15:36.375901 21981 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb031c8331e48159006ae20ce47a684" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40429 } }
I20260924 07:15:36.376014 21981 leader_election.cc:302] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdb031c8331e48159006ae20ce47a684; no voters:
I20260924 07:15:36.376175 21981 leader_election.cc:288] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:15:36.376222 21984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:15:36.376400 21981 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:15:36.376436 21984 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [term 1 LEADER]: Becoming Leader. State: Replica: fdb031c8331e48159006ae20ce47a684, State: Running, Role: LEADER
I20260924 07:15:36.376561 21984 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb031c8331e48159006ae20ce47a684" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40429 } }
I20260924 07:15:36.376680 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 21913
I20260924 07:15:36.376773 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:15:36.376878 21984 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdb031c8331e48159006ae20ce47a684. Latest consensus state: current_term: 1 leader_uuid: "fdb031c8331e48159006ae20ce47a684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb031c8331e48159006ae20ce47a684" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40429 } } }
I20260924 07:15:36.376955 21984 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [sys.catalog]: This master's current role is: LEADER
I20260924 07:15:36.376986 10763 external_mini_cluster.cc:1428] Setting key d6275738b8deae549096f041975dc0ab
I20260924 07:15:36.376888 21985 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdb031c8331e48159006ae20ce47a684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb031c8331e48159006ae20ce47a684" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 40429 } } }
I20260924 07:15:36.377116 21985 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684 [sys.catalog]: This master's current role is: LEADER
I20260924 07:15:36.378392 21981 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 07:15:36.381729 21988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:15:36.382483 21988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 07:15:36 dist-test-slave-7q7m krb5kdc[21469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:15:36.390358 21988 catalog_manager.cc:1347] Generated new cluster ID: d1819d2eab5847d4b0206c6b55510bcd
I20260924 07:15:36.390465 21988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:15:36.391812 21991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:36.377436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35250 (local address 127.10.130.254:40429)
0924 07:15:36.377590 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0924 07:15:36.377600 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:15:36.377637 (+ 37us) server_negotiation.cc:407] Connection header received
0924 07:15:36.377761 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:15:36.377768 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:15:36.377822 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:15:36.377972 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:15:36.378666 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:36.382301 (+ 3635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:36.383492 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:36.384145 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:36.384233 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:36.387175 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:15:36.387179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:15:36.387180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:15:36.387192 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:15:36.389876 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:15:36.390301 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:15:36.390304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:15:36.390304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:15:36.390407 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:15:36.390666 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:15:36.390668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:15:36.390668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:15:36.390993 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:15:36.391125 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0924 07:15:36.391411 (+ 286us) server_negotiation.cc:299] Negotiation successful
0924 07:15:36.391533 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260924 07:15:36.420629 21988 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:15:36.421411 21988 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:15:36.432889 21988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684: Generated new TSK 0
I20260924 07:15:36.433136 21988 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:15:37.124725 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.130.212:46871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:40429
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:36743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:15:37.240330 22026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:15:37.240592 22026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:15:37.240624 22026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:15:37.242244 22026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:15:37.242300 22026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:15:37.242334 22026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:15:37.244354 22026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:36743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.130.212:46871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:40429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:15:37.244587 22026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:15:37.244837 22026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:15:37.250293 22034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:15:37.250310 22032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:15:37.250478 22031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:15:37.250524 22026 server_base.cc:1034] running on GCE node
I20260924 07:15:37.250756 22026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:15:37.250976 22026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:15:37.252111 22026 hybrid_clock.cc:648] HybridClock initialized: now 1790234137252099 us; error 28 us; skew 500 ppm
Sep 24 07:15:37 dist-test-slave-7q7m krb5kdc[21469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:15:37.254547 22026 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:15:37.254937 22026 webserver.cc:460] Webserver started at http://127.10.130.193:43819/ using document root <none> and password file <none>
I20260924 07:15:37.255184 22026 fs_manager.cc:359] Metadata directory not provided
I20260924 07:15:37.255259 22026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:15:37.255417 22026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 07:15:37 dist-test-slave-7q7m krb5kdc[21469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:15:37.270462 22026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff"
format_stamp: "Formatted at 2026-09-24 07:15:37 on dist-test-slave-7q7m"
server_key: "8344b198b00f27ca138297f00289a2f4"
server_key_iv: "8f500a331130cfd4a3c9794ee329177b"
server_key_version: "kuduclusterkey@0"
I20260924 07:15:37.270927 22026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff"
format_stamp: "Formatted at 2026-09-24 07:15:37 on dist-test-slave-7q7m"
server_key: "8344b198b00f27ca138297f00289a2f4"
server_key_iv: "8f500a331130cfd4a3c9794ee329177b"
server_key_version: "kuduclusterkey@0"
I20260924 07:15:37.272696 22026 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260924 07:15:37.285789 22041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:15:37.285981 22026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:15:37.286067 22026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff"
format_stamp: "Formatted at 2026-09-24 07:15:37 on dist-test-slave-7q7m"
server_key: "8344b198b00f27ca138297f00289a2f4"
server_key_iv: "8f500a331130cfd4a3c9794ee329177b"
server_key_version: "kuduclusterkey@0"
I20260924 07:15:37.286161 22026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:15:37.315299 22026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:15:37.317128 22026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:15:37.317266 22026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:15:37.317469 22026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:15:37.317799 22026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:15:37.317873 22026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:15:37.317932 22026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:15:37.317976 22026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:15:37.324597 22026 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:42427
I20260924 07:15:37.324673 22156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:42427 every 8 connection(s)
I20260924 07:15:37.325008 22026 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
Sep 24 07:15:37 dist-test-slave-7q7m krb5kdc[21469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:15:37.331820 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 22026
I20260924 07:15:37.331920 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:15:37.332119 10763 external_mini_cluster.cc:1428] Setting key a96e9bb29a250de039a8bdda28a388de
I20260924 07:15:37.333739 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.325707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40429 (local address 127.10.130.193:44111)
0924 07:15:37.325864 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 07:15:37.325905 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 07:15:37.326352 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:15:37.326672 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:15:37.326675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:15:37.326844 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:15:37.327029 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.327033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.327677 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.327678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.328542 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.328545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.328888 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.328889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.329008 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.329451 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:15:37.329458 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:15:37.330597 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:15:37.332181 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:15:37.332187 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:15:37.332189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:15:37.332453 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:15:37.332710 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:15:37.332710 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:15:37.332711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:15:37.332846 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:15:37.333111 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:15:37.333117 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:15:37.333284 (+ 167us) client_negotiation.cc:769] Sending connection context
0924 07:15:37.333382 (+ 98us) client_negotiation.cc:240] Negotiation successful
0924 07:15:37.333523 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260924 07:15:37.334252 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.325736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:44111 (local address 127.10.130.254:40429)
0924 07:15:37.326483 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0924 07:15:37.326484 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:15:37.326490 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:15:37.326511 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:15:37.326511 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:15:37.326521 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:15:37.326575 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:15:37.327110 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.327603 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.328618 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.328825 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.328878 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.330679 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:15:37.330680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:15:37.330680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:15:37.330688 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:15:37.332084 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:15:37.332525 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:15:37.332526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:15:37.332527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:15:37.332644 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:15:37.332913 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:15:37.332914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:15:37.332914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:15:37.333040 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:15:37.333090 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 07:15:37.334071 (+ 981us) server_negotiation.cc:299] Negotiation successful
0924 07:15:37.334167 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":101,"threads_started":1}
I20260924 07:15:37.335028 22157 heartbeater.cc:347] Connected to a master server at 127.10.130.254:40429
I20260924 07:15:37.335119 22157 heartbeater.cc:464] Registering TS with master...
I20260924 07:15:37.335289 22157 heartbeater.cc:511] Master 127.10.130.254:40429 requested a full tablet report, sending...
I20260924 07:15:37.335785 21945 ts_manager.cc:194] Registered new tserver with Master: 2dbe7e7e4d2e40ae95f9a3743d7a3bff (127.10.130.193:42427)
I20260924 07:15:37.336741 21945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:44111
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:15:37.362579 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.130.212:46871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:40429
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:36743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 07:15:37 dist-test-slave-7q7m krb5kdc[21469](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790234137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260924 07:15:37.480944 22163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:15:37.481206 22163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:15:37.481243 22163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:15:37.482920 22163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:15:37.482977 22163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:15:37.483016 22163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:15:37.485127 22163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:36743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.130.212:46871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:40429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:15:37.485384 22163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:15:37.485694 22163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:15:37.491545 22170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:15:37.492379 22173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:15:37.492497 22163 server_base.cc:1034] running on GCE node
W20260924 07:15:37.494110 22171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:15:37.494359 22163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:15:37.494645 22163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:15:37.495828 22163 hybrid_clock.cc:648] HybridClock initialized: now 1790234137495725 us; error 120 us; skew 500 ppm
Sep 24 07:15:37 dist-test-slave-7q7m krb5kdc[21469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:15:37.499141 22163 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:15:37.499693 22163 webserver.cc:460] Webserver started at http://127.10.130.194:39741/ using document root <none> and password file <none>
I20260924 07:15:37.499950 22163 fs_manager.cc:359] Metadata directory not provided
I20260924 07:15:37.500041 22163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:15:37.500192 22163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 07:15:37 dist-test-slave-7q7m krb5kdc[21469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:15:37.517961 22163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "380c0525ac6c40c0bbe8af275c18dec5"
format_stamp: "Formatted at 2026-09-24 07:15:37 on dist-test-slave-7q7m"
server_key: "d2650be58e66f29657966e012a1906c1"
server_key_iv: "4509de7a12f4d426ccda516b10df4901"
server_key_version: "kuduclusterkey@0"
I20260924 07:15:37.518388 22163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "380c0525ac6c40c0bbe8af275c18dec5"
format_stamp: "Formatted at 2026-09-24 07:15:37 on dist-test-slave-7q7m"
server_key: "d2650be58e66f29657966e012a1906c1"
server_key_iv: "4509de7a12f4d426ccda516b10df4901"
server_key_version: "kuduclusterkey@0"
I20260924 07:15:37.520054 22163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260924 07:15:37.536155 22180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:15:37.536417 22163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260924 07:15:37.536516 22163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "380c0525ac6c40c0bbe8af275c18dec5"
format_stamp: "Formatted at 2026-09-24 07:15:37 on dist-test-slave-7q7m"
server_key: "d2650be58e66f29657966e012a1906c1"
server_key_iv: "4509de7a12f4d426ccda516b10df4901"
server_key_version: "kuduclusterkey@0"
I20260924 07:15:37.536631 22163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:15:37.559742 22163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:15:37.562273 22163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:15:37.562475 22163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:15:37.562811 22163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:15:37.563228 22163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:15:37.563315 22163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:15:37.563381 22163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:15:37.563429 22163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:15:37.570253 22163 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:32991
I20260924 07:15:37.570840 22163 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:15:37.572996 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 22163
I20260924 07:15:37.573096 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:15:37.573282 10763 external_mini_cluster.cc:1428] Setting key f84f21cfa44cd8bc7dbc442b00332ceb
I20260924 07:15:37.580606 22294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:32991 every 8 connection(s)
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Sep 24 07:15:37 dist-test-slave-7q7m krb5kdc[21469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:15:37.597179 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.586905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:52889 (local address 127.10.130.254:40429)
0924 07:15:37.586969 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 07:15:37.586971 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:15:37.587030 (+ 59us) server_negotiation.cc:407] Connection header received
0924 07:15:37.587717 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:15:37.587719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:15:37.587733 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:15:37.587762 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:15:37.588675 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.589456 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.591035 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.591343 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.591388 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.593943 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:15:37.593945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:15:37.593945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:15:37.593952 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:15:37.595540 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:15:37.596049 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:15:37.596050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:15:37.596051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:15:37.596122 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:15:37.596406 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:15:37.596407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:15:37.596407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:15:37.596544 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:15:37.596596 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 07:15:37.596971 (+ 375us) server_negotiation.cc:299] Negotiation successful
0924 07:15:37.597061 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 07:15:37.597685 22298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.586677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40429 (local address 127.10.130.194:52889)
0924 07:15:37.586866 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 07:15:37.586929 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0924 07:15:37.587574 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:15:37.588022 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:15:37.588027 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:15:37.588269 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:15:37.588549 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.588556 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.589573 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.589574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.590941 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.590944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.591477 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.591478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.591638 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.592256 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:15:37.592267 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:15:37.593846 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:15:37.595630 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:15:37.595637 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:15:37.595640 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:15:37.595925 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:15:37.596192 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:15:37.596193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:15:37.596193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:15:37.596329 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:15:37.596630 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:15:37.596636 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:15:37.596810 (+ 174us) client_negotiation.cc:769] Sending connection context
0924 07:15:37.597277 (+ 467us) client_negotiation.cc:240] Negotiation successful
0924 07:15:37.597439 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
I20260924 07:15:37.598138 22295 heartbeater.cc:347] Connected to a master server at 127.10.130.254:40429
I20260924 07:15:37.598241 22295 heartbeater.cc:464] Registering TS with master...
I20260924 07:15:37.598443 22295 heartbeater.cc:511] Master 127.10.130.254:40429 requested a full tablet report, sending...
I20260924 07:15:37.598788 21945 ts_manager.cc:194] Registered new tserver with Master: 380c0525ac6c40c0bbe8af275c18dec5 (127.10.130.194:32991)
I20260924 07:15:37.599278 21945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:52889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:15:37.607178 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.130.212:46871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:40429
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:36743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:15:37.714608 22300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:15:37.714957 22300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:15:37.715049 22300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:15:37.716813 22300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:15:37.716922 22300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:15:37.717006 22300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:15:37.719033 22300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:36743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.130.212:46871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:40429
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:15:37.719488 22300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:15:37.719798 22300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:15:37.726142 22308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:15:37.726316 22306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:15:37.726528 22305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:15:37.727350 22300 server_base.cc:1034] running on GCE node
I20260924 07:15:37.727542 22300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:15:37.727767 22300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:15:37.728938 22300 hybrid_clock.cc:648] HybridClock initialized: now 1790234137728916 us; error 30 us; skew 500 ppm
Sep 24 07:15:37 dist-test-slave-7q7m krb5kdc[21469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:15:37.731419 22300 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:15:37.731863 22300 webserver.cc:460] Webserver started at http://127.10.130.195:34985/ using document root <none> and password file <none>
I20260924 07:15:37.732164 22300 fs_manager.cc:359] Metadata directory not provided
I20260924 07:15:37.732228 22300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:15:37.732354 22300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 07:15:37 dist-test-slave-7q7m krb5kdc[21469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:15:37.749189 22300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "8c730adff5de4b8ea79a86c5168b4718"
format_stamp: "Formatted at 2026-09-24 07:15:37 on dist-test-slave-7q7m"
server_key: "6689f77324c06b64070cd321ef6cf465"
server_key_iv: "f7a48bdd12be8e55e1f16ad9a6890099"
server_key_version: "kuduclusterkey@0"
I20260924 07:15:37.749656 22300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "8c730adff5de4b8ea79a86c5168b4718"
format_stamp: "Formatted at 2026-09-24 07:15:37 on dist-test-slave-7q7m"
server_key: "6689f77324c06b64070cd321ef6cf465"
server_key_iv: "f7a48bdd12be8e55e1f16ad9a6890099"
server_key_version: "kuduclusterkey@0"
I20260924 07:15:37.751331 22300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:15:37.772122 22315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:15:37.772450 22300 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260924 07:15:37.772574 22300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "8c730adff5de4b8ea79a86c5168b4718"
format_stamp: "Formatted at 2026-09-24 07:15:37 on dist-test-slave-7q7m"
server_key: "6689f77324c06b64070cd321ef6cf465"
server_key_iv: "f7a48bdd12be8e55e1f16ad9a6890099"
server_key_version: "kuduclusterkey@0"
I20260924 07:15:37.772691 22300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:15:37.788794 22300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:15:37.791052 22300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:15:37.791195 22300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:15:37.791422 22300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:15:37.791766 22300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:15:37.791803 22300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:15:37.791833 22300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:15:37.791854 22300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:15:37.797919 22300 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:37417
I20260924 07:15:37.798207 22428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:37417 every 8 connection(s)
I20260924 07:15:37.798347 22300 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:15:37.806697 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 22300
I20260924 07:15:37.806802 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:15:37.806999 10763 external_mini_cluster.cc:1428] Setting key 4ca3dd590eea414e2d26f90bc546de4f
Sep 24 07:15:37 dist-test-slave-7q7m krb5kdc[21469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:15:37.811012 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.799157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:59657 (local address 127.10.130.254:40429)
0924 07:15:37.799208 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 07:15:37.799210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:15:37.800972 (+ 1762us) server_negotiation.cc:407] Connection header received
0924 07:15:37.801732 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:15:37.801734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:15:37.801746 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:15:37.801775 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:15:37.802589 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.803391 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.804907 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.805199 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.805247 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.807812 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:15:37.807813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:15:37.807814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:15:37.807821 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:15:37.809407 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:15:37.809912 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:15:37.809914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:15:37.809914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:15:37.809979 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:15:37.810266 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:15:37.810268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:15:37.810268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:15:37.810407 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:15:37.810448 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 07:15:37.810836 (+ 388us) server_negotiation.cc:299] Negotiation successful
0924 07:15:37.810919 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 07:15:37.811823 22431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.799078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:40429 (local address 127.10.130.195:59657)
0924 07:15:37.800823 (+ 1745us) negotiation.cc:107] Waiting for socket to connect
0924 07:15:37.800873 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 07:15:37.801546 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:15:37.801956 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:15:37.801961 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:15:37.802215 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:15:37.802483 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.802489 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.803501 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.803502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.804804 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.804807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.805356 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.805357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.805494 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.806088 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:15:37.806110 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:15:37.807706 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:15:37.809495 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:15:37.809501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:15:37.809503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:15:37.809815 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:15:37.810051 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:15:37.810052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:15:37.810052 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:15:37.810187 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:15:37.810493 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:15:37.810498 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:15:37.810671 (+ 173us) client_negotiation.cc:769] Sending connection context
0924 07:15:37.811409 (+ 738us) client_negotiation.cc:240] Negotiation successful
0924 07:15:37.811572 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1701,"thread_start_us":86,"threads_started":1}
I20260924 07:15:37.812453 22429 heartbeater.cc:347] Connected to a master server at 127.10.130.254:40429
I20260924 07:15:37.812543 22429 heartbeater.cc:464] Registering TS with master...
I20260924 07:15:37.812738 22429 heartbeater.cc:511] Master 127.10.130.254:40429 requested a full tablet report, sending...
I20260924 07:15:37.813154 21945 ts_manager.cc:194] Registered new tserver with Master: 8c730adff5de4b8ea79a86c5168b4718 (127.10.130.195:37417)
I20260924 07:15:37.813642 21945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:59657
I20260924 07:15:37.818554 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:15:37.832847 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.820313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34232 (local address 127.10.130.254:40429)
0924 07:15:37.820363 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:15:37.820365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:15:37.820506 (+ 141us) server_negotiation.cc:407] Connection header received
0924 07:15:37.820566 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:15:37.820567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:15:37.820578 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:15:37.820651 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:15:37.822443 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.823220 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.824319 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.824629 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.824704 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.826232 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:15:37.826234 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:15:37.826234 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:15:37.826242 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:15:37.827914 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:15:37.828321 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:15:37.828322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:15:37.828323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:15:37.828387 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:15:37.832227 (+ 3840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:15:37.832228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:15:37.832228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:15:37.832369 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:15:37.832443 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 07:15:37.832667 (+ 224us) server_negotiation.cc:299] Negotiation successful
0924 07:15:37.832750 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 07:15:37.835085 21945 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:15:37.837569 21945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:15:37.847278 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.842113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:42427 (local address 127.0.0.1:39570)
0924 07:15:37.843061 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0924 07:15:37.843071 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:15:37.843113 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:15:37.843390 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:15:37.843391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:15:37.843394 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:15:37.843439 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.843440 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.844747 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.844748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.846010 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.846013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.846988 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.846989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.847066 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.847079 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 07:15:37.847151 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 07:15:37.847201 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"thread_start_us":80,"threads_started":1}
I20260924 07:15:37.847772 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.842100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:37417 (local address 127.0.0.1:50052)
0924 07:15:37.842780 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0924 07:15:37.842821 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 07:15:37.842891 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:15:37.843353 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:15:37.843356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:15:37.843358 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:15:37.843426 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.843428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.845062 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.845063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.846211 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.846216 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.847529 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.847530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.847606 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.847610 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:15:37.847657 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 07:15:37.847696 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":68,"threads_started":1}
I20260924 07:15:37.848058 22445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.842419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39570 (local address 127.10.130.193:42427)
0924 07:15:37.843187 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0924 07:15:37.843192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:15:37.843201 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:15:37.843234 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:15:37.843236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:15:37.843251 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:15:37.843322 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:15:37.843515 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.844654 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.846103 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.846835 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.846903 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.846956 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:15:37.847802 (+ 846us) server_negotiation.cc:299] Negotiation successful
0924 07:15:37.847954 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":62,"threads_started":1}
I20260924 07:15:37.848241 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.842366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50052 (local address 127.10.130.195:37417)
0924 07:15:37.842544 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0924 07:15:37.842549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:15:37.843126 (+ 577us) server_negotiation.cc:407] Connection header received
0924 07:15:37.843175 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:15:37.843177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:15:37.843194 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:15:37.843268 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:15:37.843500 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.844695 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.846325 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.847370 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.847444 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.847497 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:15:37.848052 (+ 555us) server_negotiation.cc:299] Negotiation successful
0924 07:15:37.848092 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":75,"threads_started":1}
I20260924 07:15:37.848847 22091 tablet_service.cc:1467] Processing CreateTablet for tablet 76f50860bf574bfea5b67fa9b45cc3e4 (DEFAULT_TABLE table=test-table [id=c24c92a534c8469588429e8823a4bbfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:15:37.848886 22347 tablet_service.cc:1467] Processing CreateTablet for tablet 76f50860bf574bfea5b67fa9b45cc3e4 (DEFAULT_TABLE table=test-table [id=c24c92a534c8469588429e8823a4bbfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:15:37.849195 22091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f50860bf574bfea5b67fa9b45cc3e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:15:37.849206 22347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f50860bf574bfea5b67fa9b45cc3e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:15:37.852982 22449 tablet_bootstrap.cc:492] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718: Bootstrap starting.
I20260924 07:15:37.853932 22449 tablet_bootstrap.cc:654] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718: Neither blocks nor log segments found. Creating new log.
I20260924 07:15:37.854241 22449 log.cc:824] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718: Log is configured to *not* fsync() on all Append() calls
I20260924 07:15:37.852782 22448 tablet_bootstrap.cc:492] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff: Bootstrap starting.
I20260924 07:15:37.855171 22448 tablet_bootstrap.cc:654] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff: Neither blocks nor log segments found. Creating new log.
I20260924 07:15:37.855455 22448 log.cc:824] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff: Log is configured to *not* fsync() on all Append() calls
I20260924 07:15:37.857831 22449 tablet_bootstrap.cc:492] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718: No bootstrap required, opened a new log
I20260924 07:15:37.857900 22449 ts_tablet_manager.cc:1397] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260924 07:15:37.859800 22449 raft_consensus.cc:348] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c730adff5de4b8ea79a86c5168b4718" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37417 } } peers { permanent_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42427 } } peers { permanent_uuid: "380c0525ac6c40c0bbe8af275c18dec5" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 32991 } }
I20260924 07:15:37.859963 22449 raft_consensus.cc:374] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:15:37.860918 22449 raft_consensus.cc:729] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c730adff5de4b8ea79a86c5168b4718, State: Initialized, Role: FOLLOWER
I20260924 07:15:37.861052 22449 consensus_queue.cc:260] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c730adff5de4b8ea79a86c5168b4718" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37417 } } peers { permanent_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42427 } } peers { permanent_uuid: "380c0525ac6c40c0bbe8af275c18dec5" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 32991 } }
I20260924 07:15:37.861333 22449 ts_tablet_manager.cc:1428] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 07:15:37.861353 22429 heartbeater.cc:503] Master 127.10.130.254:40429 was elected leader, sending a full tablet report...
I20260924 07:15:37.861719 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.842631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:32991 (local address 127.0.0.1:50926)
0924 07:15:37.847695 (+ 5064us) negotiation.cc:107] Waiting for socket to connect
0924 07:15:37.847705 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:15:37.847736 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:15:37.855969 (+ 8233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:15:37.855970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:15:37.856021 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:15:37.856066 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.856068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.857479 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.857480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.858631 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.858634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.861477 (+ 2843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.861478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.861546 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.861552 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 07:15:37.861583 (+ 31us) client_negotiation.cc:240] Negotiation successful
0924 07:15:37.861645 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5054,"thread_start_us":63,"threads_started":1}
I20260924 07:15:37.862268 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.842720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50926 (local address 127.10.130.194:32991)
0924 07:15:37.855775 (+ 13055us) server_negotiation.cc:206] Beginning negotiation
0924 07:15:37.855780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:15:37.855792 (+ 12us) server_negotiation.cc:407] Connection header received
0924 07:15:37.855832 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:15:37.855834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:15:37.855852 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:15:37.855949 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:15:37.856285 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.857390 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.860054 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.860793 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.862056 (+ 1263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.862123 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 07:15:37.862151 (+ 28us) server_negotiation.cc:299] Negotiation successful
0924 07:15:37.862189 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13011,"thread_start_us":76,"threads_started":1}
I20260924 07:15:37.863045 22448 tablet_bootstrap.cc:492] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff: No bootstrap required, opened a new log
I20260924 07:15:37.863013 22229 tablet_service.cc:1467] Processing CreateTablet for tablet 76f50860bf574bfea5b67fa9b45cc3e4 (DEFAULT_TABLE table=test-table [id=c24c92a534c8469588429e8823a4bbfe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:15:37.863161 22448 ts_tablet_manager.cc:1397] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260924 07:15:37.863356 22229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76f50860bf574bfea5b67fa9b45cc3e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:15:37.865810 22448 raft_consensus.cc:348] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c730adff5de4b8ea79a86c5168b4718" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37417 } } peers { permanent_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42427 } } peers { permanent_uuid: "380c0525ac6c40c0bbe8af275c18dec5" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 32991 } }
I20260924 07:15:37.866020 22448 raft_consensus.cc:374] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:15:37.866086 22448 raft_consensus.cc:729] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dbe7e7e4d2e40ae95f9a3743d7a3bff, State: Initialized, Role: FOLLOWER
I20260924 07:15:37.866267 22448 consensus_queue.cc:260] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c730adff5de4b8ea79a86c5168b4718" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37417 } } peers { permanent_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42427 } } peers { permanent_uuid: "380c0525ac6c40c0bbe8af275c18dec5" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 32991 } }
I20260924 07:15:37.866552 22448 ts_tablet_manager.cc:1428] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:15:37.866707 22157 heartbeater.cc:503] Master 127.10.130.254:40429 was elected leader, sending a full tablet report...
I20260924 07:15:37.867062 22454 tablet_bootstrap.cc:492] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5: Bootstrap starting.
I20260924 07:15:37.867941 22454 tablet_bootstrap.cc:654] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5: Neither blocks nor log segments found. Creating new log.
I20260924 07:15:37.868258 22454 log.cc:824] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5: Log is configured to *not* fsync() on all Append() calls
I20260924 07:15:37.868991 22454 tablet_bootstrap.cc:492] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5: No bootstrap required, opened a new log
I20260924 07:15:37.869063 22454 ts_tablet_manager.cc:1397] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:15:37.870886 22454 raft_consensus.cc:348] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c730adff5de4b8ea79a86c5168b4718" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37417 } } peers { permanent_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42427 } } peers { permanent_uuid: "380c0525ac6c40c0bbe8af275c18dec5" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 32991 } }
I20260924 07:15:37.871042 22454 raft_consensus.cc:374] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:15:37.871088 22454 raft_consensus.cc:729] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380c0525ac6c40c0bbe8af275c18dec5, State: Initialized, Role: FOLLOWER
I20260924 07:15:37.871214 22454 consensus_queue.cc:260] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c730adff5de4b8ea79a86c5168b4718" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37417 } } peers { permanent_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42427 } } peers { permanent_uuid: "380c0525ac6c40c0bbe8af275c18dec5" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 32991 } }
I20260924 07:15:37.871454 22454 ts_tablet_manager.cc:1428] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260924 07:15:37.871575 22295 heartbeater.cc:503] Master 127.10.130.254:40429 was elected leader, sending a full tablet report...
Sep 24 07:15:37 dist-test-slave-7q7m krb5kdc[21469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234137, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:15:37.944680 22453 raft_consensus.cc:482] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:15:37.944824 22453 raft_consensus.cc:504] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c730adff5de4b8ea79a86c5168b4718" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37417 } } peers { permanent_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42427 } } peers { permanent_uuid: "380c0525ac6c40c0bbe8af275c18dec5" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 32991 } }
I20260924 07:15:37.945170 22453 leader_election.cc:288] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c730adff5de4b8ea79a86c5168b4718 (127.10.130.195:37417), 380c0525ac6c40c0bbe8af275c18dec5 (127.10.130.194:32991)
I20260924 07:15:37.949440 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.945866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:43395 (local address 127.10.130.194:32991)
0924 07:15:37.945906 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 07:15:37.945907 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:15:37.946118 (+ 211us) server_negotiation.cc:407] Connection header received
0924 07:15:37.946205 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:15:37.946206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:15:37.946217 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:15:37.946244 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:15:37.946446 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.947131 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.948550 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.949055 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.949107 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.949117 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 07:15:37.949326 (+ 209us) server_negotiation.cc:299] Negotiation successful
0924 07:15:37.949365 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 07:15:37.949666 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.945659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:32991 (local address 127.10.130.193:43395)
0924 07:15:37.946070 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0924 07:15:37.946078 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:15:37.946140 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:15:37.946301 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:15:37.946302 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:15:37.946303 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:15:37.946378 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.946379 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.947241 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.947242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.948452 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.948455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.949173 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.949174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.949248 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.949252 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:15:37.949527 (+ 275us) client_negotiation.cc:240] Negotiation successful
0924 07:15:37.949570 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":79,"threads_started":1}
I20260924 07:15:37.949988 22249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f50860bf574bfea5b67fa9b45cc3e4" candidate_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380c0525ac6c40c0bbe8af275c18dec5" is_pre_election: true
I20260924 07:15:37.950147 22249 raft_consensus.cc:2454] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dbe7e7e4d2e40ae95f9a3743d7a3bff in term 0.
I20260924 07:15:37.950387 22043 leader_election.cc:302] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dbe7e7e4d2e40ae95f9a3743d7a3bff, 380c0525ac6c40c0bbe8af275c18dec5; no voters:
I20260924 07:15:37.950543 22453 raft_consensus.cc:2781] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:15:37.950601 22453 raft_consensus.cc:482] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:15:37.950630 22453 raft_consensus.cc:3037] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:15:37.951489 22453 raft_consensus.cc:504] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c730adff5de4b8ea79a86c5168b4718" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37417 } } peers { permanent_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42427 } } peers { permanent_uuid: "380c0525ac6c40c0bbe8af275c18dec5" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 32991 } }
I20260924 07:15:37.951632 22453 leader_election.cc:288] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [CANDIDATE]: Term 1 election: Requested vote from peers 8c730adff5de4b8ea79a86c5168b4718 (127.10.130.195:37417), 380c0525ac6c40c0bbe8af275c18dec5 (127.10.130.194:32991)
I20260924 07:15:37.951818 22249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f50860bf574bfea5b67fa9b45cc3e4" candidate_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380c0525ac6c40c0bbe8af275c18dec5"
I20260924 07:15:37.951890 22249 raft_consensus.cc:3037] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:15:37.952522 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.945740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:46865 (local address 127.10.130.195:37417)
0924 07:15:37.945784 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 07:15:37.945786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:15:37.945963 (+ 177us) server_negotiation.cc:407] Connection header received
0924 07:15:37.946033 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:15:37.946034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:15:37.946047 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:15:37.946104 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:15:37.946317 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.947077 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.951449 (+ 4372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.952023 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.952089 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.952098 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 07:15:37.952417 (+ 319us) server_negotiation.cc:299] Negotiation successful
0924 07:15:37.952457 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 07:15:37.952732 22249 raft_consensus.cc:2454] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dbe7e7e4d2e40ae95f9a3743d7a3bff in term 1.
I20260924 07:15:37.952895 22043 leader_election.cc:302] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dbe7e7e4d2e40ae95f9a3743d7a3bff, 380c0525ac6c40c0bbe8af275c18dec5; no voters:
I20260924 07:15:37.952983 22453 raft_consensus.cc:2781] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:15:37.953100 22457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.945414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:37417 (local address 127.10.130.193:46865)
0924 07:15:37.945900 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0924 07:15:37.945911 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:15:37.945977 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:15:37.946159 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:15:37.946161 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:15:37.946163 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:15:37.946245 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.946247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.947177 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.947178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.951346 (+ 4168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:15:37.951349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.952270 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.952271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:15:37.952343 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.952346 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:15:37.952801 (+ 455us) client_negotiation.cc:240] Negotiation successful
0924 07:15:37.952844 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":2783,"thread_start_us":106,"threads_started":1}
I20260924 07:15:37.953137 22453 raft_consensus.cc:686] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [term 1 LEADER]: Becoming Leader. State: Replica: 2dbe7e7e4d2e40ae95f9a3743d7a3bff, State: Running, Role: LEADER
I20260924 07:15:37.953363 22453 consensus_queue.cc:237] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c730adff5de4b8ea79a86c5168b4718" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37417 } } peers { permanent_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42427 } } peers { permanent_uuid: "380c0525ac6c40c0bbe8af275c18dec5" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 32991 } }
I20260924 07:15:37.953564 22378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f50860bf574bfea5b67fa9b45cc3e4" candidate_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c730adff5de4b8ea79a86c5168b4718" is_pre_election: true
I20260924 07:15:37.953714 22378 raft_consensus.cc:2454] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dbe7e7e4d2e40ae95f9a3743d7a3bff in term 0.
I20260924 07:15:37.953806 22379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76f50860bf574bfea5b67fa9b45cc3e4" candidate_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c730adff5de4b8ea79a86c5168b4718"
I20260924 07:15:37.953855 22379 raft_consensus.cc:3037] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:15:37.954188 21945 catalog_manager.cc:5495] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff reported cstate change: term changed from 0 to 1, leader changed from <none> to 2dbe7e7e4d2e40ae95f9a3743d7a3bff (127.10.130.193). New cstate: current_term: 1 leader_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c730adff5de4b8ea79a86c5168b4718" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2dbe7e7e4d2e40ae95f9a3743d7a3bff" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 42427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "380c0525ac6c40c0bbe8af275c18dec5" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 32991 } health_report { overall_health: UNKNOWN } } }
I20260924 07:15:37.954604 22379 raft_consensus.cc:2454] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dbe7e7e4d2e40ae95f9a3743d7a3bff in term 1.
I20260924 07:15:37.972291 22445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:15:37.967514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39584 (local address 127.10.130.193:42427)
0924 07:15:37.967552 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 07:15:37.967554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:15:37.967561 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:15:37.967607 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:15:37.967607 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:15:37.967628 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:15:37.967686 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:15:37.967822 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.968510 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.971362 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:15:37.971621 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:15:37.971688 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:15:37.971900 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:15:37.972074 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:15:37.972154 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 07:15:37.972172 (+ 18us) server_negotiation.cc:299] Negotiation successful
0924 07:15:37.972195 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:15:37.975615 22249 raft_consensus.cc:1261] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5 [term 1 FOLLOWER]: Refusing update from remote peer 2dbe7e7e4d2e40ae95f9a3743d7a3bff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:15:37.975615 22379 raft_consensus.cc:1261] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718 [term 1 FOLLOWER]: Refusing update from remote peer 2dbe7e7e4d2e40ae95f9a3743d7a3bff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:15:37.976033 22459 consensus_queue.cc:1035] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c730adff5de4b8ea79a86c5168b4718" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:15:37.976215 22459 consensus_queue.cc:1035] T 76f50860bf574bfea5b67fa9b45cc3e4 P 2dbe7e7e4d2e40ae95f9a3743d7a3bff [LEADER]: Connected to new peer: Peer: permanent_uuid: "380c0525ac6c40c0bbe8af275c18dec5" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 32991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:15:37.981348 22463 mvcc.cc:205] Tried to move back new op lower bound from 7332799029142159360 to 7332799029057699840. Current Snapshot: MvccSnapshot[applied={T|T < 7332799029142159360}]
I20260924 07:15:37.983489 22464 mvcc.cc:205] Tried to move back new op lower bound from 7332799029142159360 to 7332799029057699840. Current Snapshot: MvccSnapshot[applied={T|T < 7332799029142159360}]
I20260924 07:15:37.990201 21945 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:15:37.990304 21945 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:15:37.991032 21945 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdb031c8331e48159006ae20ce47a684: Sending DeleteTablet for 3 replicas of tablet 76f50860bf574bfea5b67fa9b45cc3e4
I20260924 07:15:37.991427 22347 tablet_service.cc:1514] Processing DeleteTablet for tablet 76f50860bf574bfea5b67fa9b45cc3e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:15:37 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:50052
I20260924 07:15:37.991669 22091 tablet_service.cc:1514] Processing DeleteTablet for tablet 76f50860bf574bfea5b67fa9b45cc3e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:15:37 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:39570
I20260924 07:15:37.992033 22229 tablet_service.cc:1514] Processing DeleteTablet for tablet 76f50860bf574bfea5b67fa9b45cc3e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:15:37 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:50926
I20260924 07:15:37.992709 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 22026
I20260924 07:15:37.994944 22474 tablet_replica.cc:330] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5: stopping tablet replica
I20260924 07:15:37.995132 22474 raft_consensus.cc:2229] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:15:37.995332 22474 raft_consensus.cc:2258] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:15:37.995544 22472 tablet_replica.cc:330] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718: stopping tablet replica
I20260924 07:15:37.995707 22472 raft_consensus.cc:2229] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:15:37.995875 22472 raft_consensus.cc:2258] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:15:37.995895 22474 ts_tablet_manager.cc:1905] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:15:37.996423 22472 ts_tablet_manager.cc:1905] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:15:37.997881 22472 ts_tablet_manager.cc:1918] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:15:37.997999 22472 log.cc:1196] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/76f50860bf574bfea5b67fa9b45cc3e4
I20260924 07:15:37.998574 22472 ts_tablet_manager.cc:1939] T 76f50860bf574bfea5b67fa9b45cc3e4 P 8c730adff5de4b8ea79a86c5168b4718: Deleting consensus metadata
I20260924 07:15:37.999010 21930 catalog_manager.cc:4841] TS 8c730adff5de4b8ea79a86c5168b4718 (127.10.130.195:37417): tablet 76f50860bf574bfea5b67fa9b45cc3e4 (table test-table [id=c24c92a534c8469588429e8823a4bbfe]) successfully deleted
I20260924 07:15:38.002049 22474 ts_tablet_manager.cc:1918] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:15:38.002128 22474 log.cc:1196] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/76f50860bf574bfea5b67fa9b45cc3e4
I20260924 07:15:38.003245 22474 ts_tablet_manager.cc:1939] T 76f50860bf574bfea5b67fa9b45cc3e4 P 380c0525ac6c40c0bbe8af275c18dec5: Deleting consensus metadata
I20260924 07:15:38.003695 21929 catalog_manager.cc:4841] TS 380c0525ac6c40c0bbe8af275c18dec5 (127.10.130.194:32991): tablet 76f50860bf574bfea5b67fa9b45cc3e4 (table test-table [id=c24c92a534c8469588429e8823a4bbfe]) successfully deleted
W20260924 07:15:38.003997 21931 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:42427 (error 108)
I20260924 07:15:38.004596 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 22163
I20260924 07:15:38.010067 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 22300
I20260924 07:15:38.015148 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 21913
I20260924 07:15:38.021070 10763 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 07:15:38.355404 10763 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 07:15:38.355538 10763 mini_ranger.cc:66] Stopping Ranger...
I20260924 07:15:38.689050 10763 mini_ranger.cc:68] Stopped Ranger
2026-09-24 07:15:38.689 UTC [21499] LOG: received smart shutdown request
2026-09-24 07:15:38.692 UTC [21499] LOG: background worker "logical replication launcher" (PID 21508) exited with exit code 1
2026-09-24 07:15:38.692 UTC [21503] LOG: shutting down
2026-09-24 07:15:38.825 UTC [21499] LOG: database system is shut down
2026-09-24T07:15:38Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (42645 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:15:38 dist-test-slave-7q7m krb5kdc[22484](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:15:38 dist-test-slave-7q7m krb5kdc[22484](info): set up 2 sockets
Sep 24 07:15:38 dist-test-slave-7q7m krb5kdc[22484](info): commencing operation
krb5kdc: starting...
W20260924 07:15:39.855603 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:15:39 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:15:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:15:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.130.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.130.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/rangeradmin_127.10.130.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/rangeradmin_127.10.130.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.130.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.130.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/rangerlookup_127.10.130.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/rangerlookup_127.10.130.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.130.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.130.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/HTTP_127.10.130.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/HTTP_127.10.130.212@KRBTEST.COM.keytab.
I20260924 07:15:39.996519 10763 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task9LiIRP/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/postgres -l logfile start
2026-09-24 07:15:42.084 UTC [22514] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 07:15:42.084 UTC [22514] LOG: listening on IPv4 address "127.10.130.212", port 33337
2026-09-24 07:15:42.087 UTC [22514] LOG: listening on Unix socket "/tmp/.s.PGSQL.33337"
2026-09-24 07:15:42.111 UTC [22517] LOG: database system was shut down at 2026-09-24 07:15:40 UTC
2026-09-24 07:15:42.114 UTC [22514] LOG: database system is ready to accept connections
I20260924 07:15:42.971446 10763 mini_postgres.cc:96] Postgres bound to 33337
2026-09-24 07:15:42.977 UTC [22525] FATAL: database "slave" does not exist
127.10.130.212:33337 - accepting connections
I20260924 07:15:42.977480 10763 mini_ranger.cc:161] Starting Ranger...
I20260924 07:15:42.990831 10763 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 07:15:43.357734 10763 mini_ranger.cc:87] Created ranger Postgres database
I20260924 07:15:43.357810 10763 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-admin
2026-09-24 07:15:43,624 [I] DB FLAVOR :POSTGRES
2026-09-24 07:15:43,625 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 07:15:43,625 [I] Checking connection..
2026-09-24 07:15:43,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 07:15:43,843 [I] Checking connection passed.
2026-09-24 07:15:43,843 [I] --------- Verifying version history table ---------
2026-09-24 07:15:43,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 07:15:44,089 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 07:15:44,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 07:15:44,305 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 07:15:44,306 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 07:15:44,306 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 07:15:44.539 UTC [22589] WARNING: there is no transaction in progress
2026-09-24 07:15:44,544 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 07:15:44,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 07:15:44,784 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 07:15:44,784 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 07:15:44,784 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 07:15:45,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 07:15:45,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7q7m.c.gcp-upstream.internal','N') ;"
2026-09-24 07:15:45,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 07:15:45,722 [I] Table x_portal_user does not exist in database ranger
2026-09-24 07:15:45,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 07:15:45,971 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 07:15:45,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 07:15:46,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 07:15:46,431 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 07:15:46,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 07:15:47.302 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.316 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.330 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.343 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.359 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.374 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.424 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.429 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.442 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.451 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.460 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.472 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.480 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.486 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.492 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:47.497 UTC [22697] WARNING: there is no transaction in progress
2026-09-24 07:15:48,010 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 07:15:48,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 07:15:48,233 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 07:15:48,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 07:15:48,472 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 07:15:48,473 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 07:15:48,721 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 07:15:48,721 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 07:15:48,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 07:15:49,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7q7m.c.gcp-upstream.internal';"
2026-09-24 07:15:49,413 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 07:15:49,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 07:15:49,660 [I] DB_PATCHES have already been applied
I20260924 07:15:49.665472 10763 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-admin:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task9LiIRP/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 07:15:49.665531 10763 mini_ranger.cc:193] Using host: 127.10.130.212
I20260924 07:15:49.668290 10763 mini_ranger.cc:227] Ranger admin URL: http://127.10.130.212:34887
Sep 24, 2026 7:15:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 7:15:49 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 7:15:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 7:15:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 7:15:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 7:15:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 7:15:50 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 07:15:50.652966 10763 mini_ranger.cc:160] Time spent starting Ranger: real 7.675s user 0.000s sys 0.007s
Sep 24, 2026 7:16:09 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 7:16:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 24, 2026 7:16:09 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 7:16:09 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 7:16:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 7:16:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 7:16:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 7:16:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 7:16:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 7:16:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 7:16:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 7:16:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 7:16:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 7:16:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 7:16:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 7:16:09 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 7:16:09 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 7:16:10 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 07:16:10.891999 10763 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.130.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.130.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/rangerkms_127.10.130.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.130.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/rangerkms_127.10.130.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.130.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.130.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.130.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/HTTP_127.10.130.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.130.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/HTTP_127.10.130.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 07:16:10 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234170, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 07:16:10.964031 10763 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 07:16:10.972327 10763 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 07:16:11.700595 10763 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 07:16:11.700670 10763 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-kms
I20260924 07:16:11.700682 10763 mini_ranger_kms.cc:227] Using postgres at 127.10.130.212:33337
2026-09-24 07:16:11,845 [I] DB FLAVOR :POSTGRES
2026-09-24 07:16:11,846 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 07:16:11,846 [I] Checking connection
2026-09-24 07:16:11,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 07:16:12,088 [I] connection success
2026-09-24 07:16:12,088 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 07:16:12,088 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 07:16:12,089 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 07:16:12,342 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 07:16:12,342 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 07:16:12,342 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 07:16:12,342 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.130.212:33337/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 07:16:12,598 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 07:16:13.027352 10763 mini_ranger_kms.cc:310] Created kms service
I20260924 07:16:13.210338 10763 mini_ranger_kms.cc:326] Created kudu user
I20260924 07:16:13.250358 10763 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 07:16:13 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234170, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:16:13.632477 10763 mini_ranger_kms.cc:384] Added ranger policy
I20260924 07:16:13.632592 10763 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-kms:/tmp/dist-test-task9LiIRP/build/release/bin/postgresql.jar:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task9LiIRP/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 07:16:13.632611 10763 mini_ranger_kms.cc:240] Using host: 127.10.130.212
I20260924 07:16:13.635494 10763 mini_ranger_kms.cc:276] Ranger KMS PID: 22877
I20260924 07:16:13.635581 10763 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.130.212:54751
Sep 24, 2026 7:16:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 7:16:14 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 7:16:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 7:16:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 7:16:15 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 7:16:15 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260924 07:16:15.625835 10763 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.662s user 0.000s sys 0.128s
I20260924 07:16:15.625942 10763 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 07:16:15.625988 10763 mini_ranger_kms.cc:401] 127.10.130.212:54751/kms/v1/keys
Sep 24 07:16:15 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790234175, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.130.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 24 07:16:16 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234175, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.130.212@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
Sep 24 07:16:16 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234175, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.130.212@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
Sep 24 07:16:18 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:16:18.794579 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:38423
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:38423
--ranger_config_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.130.212:54751/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:18.872018 22928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:18.872226 22928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:18.872254 22928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:18.873410 22928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:16:18.873450 22928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:18.873467 22928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:18.873482 22928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:16:18.873497 22928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:16:18.875013 22928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.130.212:54751/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:38423
--ranger_config_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:38423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:18.875236 22928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:18.875419 22928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:18.880023 22933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:18.880108 22936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:18.880110 22934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:18.880108 22928 server_base.cc:1034] running on GCE node
I20260924 07:16:18.880581 22928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:18.880853 22928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:18.881989 22928 hybrid_clock.cc:648] HybridClock initialized: now 1790234178881973 us; error 32 us; skew 500 ppm
Sep 24 07:16:18 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234178, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:18.888788 22928 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:16:18.889216 22928 webserver.cc:460] Webserver started at http://127.10.130.254:40535/ using document root <none> and password file <none>
I20260924 07:16:18.889429 22928 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:18.889467 22928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:18.889573 22928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 07:16:18 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234178, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:16:18.940115 22928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "01e79185d5b64ef99f57d815a8bd8ccf"
format_stamp: "Formatted at 2026-09-24 07:16:18 on dist-test-slave-7q7m"
server_key: "53d9f432ddbfa76b8e30dab98045bcd3"
server_key_iv: "81d6668ec8c21d0f5c59000e8cb25dc9"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:18.940496 22928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "01e79185d5b64ef99f57d815a8bd8ccf"
format_stamp: "Formatted at 2026-09-24 07:16:18 on dist-test-slave-7q7m"
server_key: "53d9f432ddbfa76b8e30dab98045bcd3"
server_key_iv: "81d6668ec8c21d0f5c59000e8cb25dc9"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:18.941746 22928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:16:18.956152 22943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:18.956311 22928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:16:18.956401 22928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "01e79185d5b64ef99f57d815a8bd8ccf"
format_stamp: "Formatted at 2026-09-24 07:16:18 on dist-test-slave-7q7m"
server_key: "53d9f432ddbfa76b8e30dab98045bcd3"
server_key_iv: "81d6668ec8c21d0f5c59000e8cb25dc9"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:18.956498 22928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:18.972954 22928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:18.974726 22928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:18.974856 22928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:18.979161 22928 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:38423
I20260924 07:16:18.979189 22995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:38423 every 8 connection(s)
I20260924 07:16:18.979558 22928 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:16:18.980320 22996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:18.982460 22996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf: Bootstrap starting.
I20260924 07:16:18.983009 22996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:18.983273 22996 log.cc:824] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:18.983867 22996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf: No bootstrap required, opened a new log
I20260924 07:16:18.985800 22996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 38423 } }
I20260924 07:16:18.985936 22996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:18.985965 22996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e79185d5b64ef99f57d815a8bd8ccf, State: Initialized, Role: FOLLOWER
I20260924 07:16:18.986089 22996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 38423 } }
I20260924 07:16:18.986157 22996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:16:18.986181 22996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:16:18.986224 22996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:18.986766 22996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 38423 } }
I20260924 07:16:18.986856 22996 leader_election.cc:302] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01e79185d5b64ef99f57d815a8bd8ccf; no voters:
I20260924 07:16:18.987011 22996 leader_election.cc:288] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:16:18.987047 22999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:16:18.987191 22999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 1 LEADER]: Becoming Leader. State: Replica: 01e79185d5b64ef99f57d815a8bd8ccf, State: Running, Role: LEADER
I20260924 07:16:18.987264 22996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:16:18.987322 22999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 38423 } }
I20260924 07:16:18.987659 22999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e79185d5b64ef99f57d815a8bd8ccf. Latest consensus state: current_term: 1 leader_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 38423 } } }
I20260924 07:16:18.987670 23001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 38423 } } }
I20260924 07:16:18.987830 22999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:18.987841 23001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:18.988073 23008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:16:18.988762 23008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:16:18.988994 22996 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 07:16:18.989784 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 22928
I20260924 07:16:18.989874 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:16:18.990053 10763 external_mini_cluster.cc:1428] Setting key 79f3de18f7958d41a41af093aa6f96f9
I20260924 07:16:18.993686 23008 catalog_manager.cc:1347] Generated new cluster ID: 19b0a6e862704acea25698d8cda35fb3
I20260924 07:16:18.993788 23008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:16:19.004242 23008 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:16:19.004990 23008 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:16:19.010716 23008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf: Generated new TSK 0
I20260924 07:16:19.010932 23008 catalog_manager.cc:1514] Initializing in-progress tserver states...
Sep 24 07:16:19 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:19.017885 23019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:18.991165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39736 (local address 127.10.130.254:38423)
0924 07:16:18.992591 (+ 1426us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:18.992604 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:18.996775 (+ 4171us) server_negotiation.cc:407] Connection header received
0924 07:16:18.996935 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:18.996942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:18.997008 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:18.997116 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:18.998865 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:19.000292 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:19.009327 (+ 9035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:19.010034 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:19.010172 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:19.013151 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:19.013156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:19.013157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:19.013170 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:19.015775 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:19.016223 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:19.016227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:19.016227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:19.016347 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:19.016558 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:19.016558 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:19.016559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:19.016855 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:19.017430 (+ 575us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:19.017505 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 07:16:19.017659 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1334,"thread_start_us":187,"threads_started":1}
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:16:19.782513 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.130.212:54751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:38423
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:19.888644 23041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:19.888828 23041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:19.888849 23041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:19.889958 23041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:19.889999 23041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:19.890033 23041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:16:19.891439 23041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.130.212:54751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:38423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:19.891667 23041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:19.891919 23041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:19.896667 23047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:19.896648 23049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:19.897079 23046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:19.897430 23041 server_base.cc:1034] running on GCE node
I20260924 07:16:19.897625 23041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:19.897861 23041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:19.898993 23041 hybrid_clock.cc:648] HybridClock initialized: now 1790234179898976 us; error 34 us; skew 500 ppm
Sep 24 07:16:19 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234179, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:19.901624 23041 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:16:19.902153 23041 webserver.cc:460] Webserver started at http://127.10.130.193:39911/ using document root <none> and password file <none>
I20260924 07:16:19.902402 23041 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:19.902469 23041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:19.902608 23041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 07:16:19 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234179, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:16:19.916774 23041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "cce9187c8b8e4dfd9a5386fc1a948122"
format_stamp: "Formatted at 2026-09-24 07:16:19 on dist-test-slave-7q7m"
server_key: "275fcaaf64d26fdbe5cd4348fbfda6fc"
server_key_iv: "2ce8081d34fbcbe09258087b7870ac18"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:19.917271 23041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "cce9187c8b8e4dfd9a5386fc1a948122"
format_stamp: "Formatted at 2026-09-24 07:16:19 on dist-test-slave-7q7m"
server_key: "275fcaaf64d26fdbe5cd4348fbfda6fc"
server_key_iv: "2ce8081d34fbcbe09258087b7870ac18"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:19.919364 23041 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260924 07:16:19.933445 23056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:19.933959 23041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260924 07:16:19.934067 23041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "cce9187c8b8e4dfd9a5386fc1a948122"
format_stamp: "Formatted at 2026-09-24 07:16:19 on dist-test-slave-7q7m"
server_key: "275fcaaf64d26fdbe5cd4348fbfda6fc"
server_key_iv: "2ce8081d34fbcbe09258087b7870ac18"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:19.934158 23041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:19.949714 23041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:19.952049 23041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:19.952194 23041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:19.952431 23041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:19.952801 23041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:19.952855 23041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:19.952886 23041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:19.952943 23041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:19.960297 23041 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:41717
I20260924 07:16:19.960440 23171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:41717 every 8 connection(s)
I20260924 07:16:19.960821 23041 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:16:19.967788 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 23041
I20260924 07:16:19.967871 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:16:19.968091 10763 external_mini_cluster.cc:1428] Setting key 0d75e0854ef845f1cfe76962d1d78cd6
Sep 24 07:16:19 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234179, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:19.972671 23175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:19.961685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:38423 (local address 127.10.130.193:51769)
0924 07:16:19.961883 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:19.961942 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:19.962592 (+ 650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:19.962982 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:19.962986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:19.963243 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:19.963660 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:19.963668 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:19.964726 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:19.964727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:19.966006 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:19.966009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:19.966553 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:19.966554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:19.966688 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:19.967272 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:19.967283 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:19.969472 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:19.971123 (+ 1651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:19.971130 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:19.971132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:19.971391 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:19.971580 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:19.971581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:19.971581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:19.971697 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:19.971953 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:19.971959 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:19.972149 (+ 190us) client_negotiation.cc:769] Sending connection context
0924 07:16:19.972256 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 07:16:19.972416 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":83,"threads_started":1}
I20260924 07:16:19.973409 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:19.961680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:51769 (local address 127.10.130.254:38423)
0924 07:16:19.961811 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:19.961813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:19.962028 (+ 215us) server_negotiation.cc:407] Connection header received
0924 07:16:19.962718 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:19.962720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:19.962736 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:19.962780 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:19.963767 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:19.964617 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:19.966103 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:19.966420 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:19.966459 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:19.969568 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:19.969569 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:19.969569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:19.969577 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:19.971041 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:19.971465 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:19.971466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:19.971467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:19.971526 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:19.971753 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:19.971754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:19.971754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:19.971868 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:19.971930 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:19.973223 (+ 1293us) server_negotiation.cc:299] Negotiation successful
0924 07:16:19.973312 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260924 07:16:19.973759 23172 heartbeater.cc:347] Connected to a master server at 127.10.130.254:38423
I20260924 07:16:19.973881 23172 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:19.974097 23172 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
I20260924 07:16:19.974694 22960 ts_manager.cc:194] Registered new tserver with Master: cce9187c8b8e4dfd9a5386fc1a948122 (127.10.130.193:41717)
I20260924 07:16:19.975589 22960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:51769
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:16:19.992815 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.130.212:54751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:38423
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:20.099349 23178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:20.099671 23178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:20.099759 23178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:20.101445 23178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:20.101543 23178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:20.101601 23178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:16:20.103613 23178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.130.212:54751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:38423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:20.103848 23178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:20.104183 23178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260924 07:16:20.110576 23186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:20.110682 23178 server_base.cc:1034] running on GCE node
W20260924 07:16:20.110891 23188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:20.111096 23185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:20.111413 23178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:20.111716 23178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:20.112886 23178 hybrid_clock.cc:648] HybridClock initialized: now 1790234180112856 us; error 41 us; skew 500 ppm
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:20.116248 23178 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:16:20.116847 23178 webserver.cc:460] Webserver started at http://127.10.130.194:33039/ using document root <none> and password file <none>
I20260924 07:16:20.117117 23178 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:20.117179 23178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:20.117357 23178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:16:20.136171 23178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "a5fb374b7df1451a845f63eb2c1052ad"
format_stamp: "Formatted at 2026-09-24 07:16:20 on dist-test-slave-7q7m"
server_key: "b71b7a4f926048a6d02f6ebcce574896"
server_key_iv: "27486539d5d2a0e112e5c5f340dae476"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:20.136634 23178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "a5fb374b7df1451a845f63eb2c1052ad"
format_stamp: "Formatted at 2026-09-24 07:16:20 on dist-test-slave-7q7m"
server_key: "b71b7a4f926048a6d02f6ebcce574896"
server_key_iv: "27486539d5d2a0e112e5c5f340dae476"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:20.138612 23178 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 07:16:20.152279 23195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:20.152513 23178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260924 07:16:20.152601 23178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "a5fb374b7df1451a845f63eb2c1052ad"
format_stamp: "Formatted at 2026-09-24 07:16:20 on dist-test-slave-7q7m"
server_key: "b71b7a4f926048a6d02f6ebcce574896"
server_key_iv: "27486539d5d2a0e112e5c5f340dae476"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:20.152685 23178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:20.169881 23178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:20.172178 23178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:20.172317 23178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:20.172571 23178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:20.172993 23178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:20.173043 23178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:20.173074 23178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:20.173089 23178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:20.179498 23178 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:43583
I20260924 07:16:20.179567 23308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:43583 every 8 connection(s)
I20260924 07:16:20.179852 23178 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:20.188428 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 23178
I20260924 07:16:20.188526 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:16:20.188724 10763 external_mini_cluster.cc:1428] Setting key 9d315065b84a628cfa054496e47d62bc
I20260924 07:16:20.191887 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:20.180616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:44889 (local address 127.10.130.254:38423)
0924 07:16:20.180668 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:20.180670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:20.180882 (+ 212us) server_negotiation.cc:407] Connection header received
0924 07:16:20.181382 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:20.181384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:20.181400 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:20.181476 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:20.182044 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:20.182811 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:20.183892 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:20.184230 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:20.184311 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:20.186529 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:20.186530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:20.186530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:20.186538 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:20.188225 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:20.189308 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:20.189309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:20.189309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:20.189371 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:20.191159 (+ 1788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:20.191160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:20.191161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:20.191296 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:20.191335 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:20.191704 (+ 369us) server_negotiation.cc:299] Negotiation successful
0924 07:16:20.191793 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 07:16:20.192387 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:20.180618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:38423 (local address 127.10.130.194:44889)
0924 07:16:20.180766 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:20.180802 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:20.181267 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:20.181535 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:20.181538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:20.181708 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:20.181929 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:20.181936 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:20.182917 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:20.182918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:20.183801 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:20.183804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:20.184317 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:20.184318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:20.184425 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:20.184973 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:20.184981 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:20.186418 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:20.188928 (+ 2510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:20.188934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:20.188936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:20.189219 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:20.190936 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:20.190937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:20.190938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:20.191079 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:20.191378 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:20.191384 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:20.191540 (+ 156us) client_negotiation.cc:769] Sending connection context
0924 07:16:20.191999 (+ 459us) client_negotiation.cc:240] Negotiation successful
0924 07:16:20.192156 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260924 07:16:20.192919 23309 heartbeater.cc:347] Connected to a master server at 127.10.130.254:38423
I20260924 07:16:20.193022 23309 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:20.193202 23309 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
I20260924 07:16:20.193555 22960 ts_manager.cc:194] Registered new tserver with Master: a5fb374b7df1451a845f63eb2c1052ad (127.10.130.194:43583)
I20260924 07:16:20.194037 22960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:44889
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:16:20.220104 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.130.212:54751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:38423
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:20.334201 23315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:20.334479 23315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:20.334513 23315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:20.336266 23315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:20.336321 23315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:20.336359 23315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:16:20.338487 23315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.130.212:54751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:38423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:20.338728 23315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:20.339033 23315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:20.345101 23323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:20.345264 23320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:20.345208 23321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:20.345098 23315 server_base.cc:1034] running on GCE node
I20260924 07:16:20.346511 23315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:20.346781 23315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:20.347950 23315 hybrid_clock.cc:648] HybridClock initialized: now 1790234180347936 us; error 29 us; skew 500 ppm
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:20.350883 23315 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:16:20.351408 23315 webserver.cc:460] Webserver started at http://127.10.130.195:41079/ using document root <none> and password file <none>
I20260924 07:16:20.351656 23315 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:20.351716 23315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:20.351845 23315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:16:20.365628 23315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "5158fffa2db5426c8db6ceb3774e6205"
format_stamp: "Formatted at 2026-09-24 07:16:20 on dist-test-slave-7q7m"
server_key: "e585e268c9c50577f460bc4f7bd8c9f5"
server_key_iv: "653aa10a0b10842c4feec61479002e71"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:20.366074 23315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "5158fffa2db5426c8db6ceb3774e6205"
format_stamp: "Formatted at 2026-09-24 07:16:20 on dist-test-slave-7q7m"
server_key: "e585e268c9c50577f460bc4f7bd8c9f5"
server_key_iv: "653aa10a0b10842c4feec61479002e71"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:20.378575 23315 fs_manager.cc:691] Time spent creating directory manager: real 0.012s user 0.003s sys 0.001s
I20260924 07:16:20.392908 23330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:20.393137 23315 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:16:20.393227 23315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "5158fffa2db5426c8db6ceb3774e6205"
format_stamp: "Formatted at 2026-09-24 07:16:20 on dist-test-slave-7q7m"
server_key: "e585e268c9c50577f460bc4f7bd8c9f5"
server_key_iv: "653aa10a0b10842c4feec61479002e71"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:20.393319 23315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:20.408769 23315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:20.411006 23315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:20.411191 23315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:20.411418 23315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:20.411840 23315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:20.411916 23315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:20.411969 23315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:20.412048 23315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:20.418656 23315 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:46045
I20260924 07:16:20.419114 23315 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:16:20.427058 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 23315
I20260924 07:16:20.427160 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:16:20.427340 10763 external_mini_cluster.cc:1428] Setting key cfafc842e3ef2f5dde4a966551f2e3df
I20260924 07:16:20.429817 23443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:46045 every 8 connection(s)
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:20.442663 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:20.431629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:56367 (local address 127.10.130.254:38423)
0924 07:16:20.431872 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:20.431874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:20.432062 (+ 188us) server_negotiation.cc:407] Connection header received
0924 07:16:20.432870 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:20.432871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:20.432883 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:20.432912 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:20.433722 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:20.434506 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:20.435928 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:20.436248 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:20.436294 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:20.439359 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:20.439360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:20.439361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:20.439368 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:20.440953 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:20.441543 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:20.441544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:20.441545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:20.441605 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:20.441897 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:20.441898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:20.441898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:20.442032 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:20.442090 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:20.442459 (+ 369us) server_negotiation.cc:299] Negotiation successful
0924 07:16:20.442547 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220}
I20260924 07:16:20.443374 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:20.431625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:38423 (local address 127.10.130.195:56367)
0924 07:16:20.431793 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:20.431835 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:20.432753 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:20.433109 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:20.433114 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:20.433350 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:20.433618 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:20.433625 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:20.434615 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:20.434616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:20.435827 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:20.435830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:20.436399 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:20.436400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:20.436535 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:20.437397 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:20.437406 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:20.439273 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:20.441045 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:20.441051 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:20.441054 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:20.441452 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:20.441678 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:20.441679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:20.441679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:20.441814 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:20.442139 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:20.442146 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:20.442301 (+ 155us) client_negotiation.cc:769] Sending connection context
0924 07:16:20.442880 (+ 579us) client_negotiation.cc:240] Negotiation successful
0924 07:16:20.443037 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260924 07:16:20.443799 23444 heartbeater.cc:347] Connected to a master server at 127.10.130.254:38423
I20260924 07:16:20.443889 23444 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:20.444125 23444 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
I20260924 07:16:20.444519 22960 ts_manager.cc:194] Registered new tserver with Master: 5158fffa2db5426c8db6ceb3774e6205 (127.10.130.195:46045)
I20260924 07:16:20.444993 22960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:56367
I20260924 07:16:20.454200 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:16:20.454929 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 23041
I20260924 07:16:20.461652 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 23178
I20260924 07:16:20.467780 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 23315
I20260924 07:16:20.474018 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 22928
I20260924 07:16:20.482761 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:38423
--webserver_interface=127.10.130.254
--webserver_port=40535
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:38423
--ranger_config_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.130.212:54751/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.130.212:54751/kms with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:20.566958 23447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:20.567152 23447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:20.567181 23447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:20.568310 23447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:16:20.568348 23447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:20.568368 23447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:20.568383 23447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:16:20.568406 23447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:16:20.569947 23447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.130.212:54751/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:38423
--ranger_config_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:38423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=40535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:20.570173 23447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:20.570369 23447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:20.574952 23452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:20.574908 23453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:20.575094 23447 server_base.cc:1034] running on GCE node
W20260924 07:16:20.575085 23455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:20.575548 23447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:20.575870 23447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:20.577013 23447 hybrid_clock.cc:648] HybridClock initialized: now 1790234180576991 us; error 45 us; skew 500 ppm
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:20.579870 23447 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:16:20.580420 23447 webserver.cc:460] Webserver started at http://127.10.130.254:40535/ using document root <none> and password file <none>
I20260924 07:16:20.580643 23447 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:20.580703 23447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:16:20.621825 23447 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:16:20.622587 23463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:20.622758 23447 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:16:20.622853 23447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "01e79185d5b64ef99f57d815a8bd8ccf"
format_stamp: "Formatted at 2026-09-24 07:16:18 on dist-test-slave-7q7m"
server_key: "53d9f432ddbfa76b8e30dab98045bcd3"
server_key_iv: "81d6668ec8c21d0f5c59000e8cb25dc9"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:20.623317 23447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:20.632893 23447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:20.634764 23447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:20.634902 23447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:20.639724 23447 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:38423
I20260924 07:16:20.639753 23515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:38423 every 8 connection(s)
I20260924 07:16:20.640230 23447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:16:20.641424 23516 sys_catalog.cc:263] Verifying existing consensus state
I20260924 07:16:20.642058 23516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf: Bootstrap starting.
I20260924 07:16:20.648936 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 23447
I20260924 07:16:20.649266 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.130.212:54751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:41717
--local_ip_for_outbound_sockets=127.10.130.193
--tserver_master_addrs=127.10.130.254:38423
--webserver_port=39911
--webserver_interface=127.10.130.193
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.130.212:54751/kms with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:16:20.649675 23516 log.cc:824] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:20.651490 23516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 07:16:20.651733 23516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf: Bootstrap complete.
I20260924 07:16:20.653961 23516 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 38423 } }
I20260924 07:16:20.654202 23516 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e79185d5b64ef99f57d815a8bd8ccf, State: Initialized, Role: FOLLOWER
I20260924 07:16:20.654408 23516 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 38423 } }
I20260924 07:16:20.654474 23516 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:16:20.654536 23516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:16:20.654588 23516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 1 FOLLOWER]: Advancing to term 2
I20260924 07:16:20.655555 23516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 38423 } }
I20260924 07:16:20.655647 23516 leader_election.cc:302] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01e79185d5b64ef99f57d815a8bd8ccf; no voters:
I20260924 07:16:20.655817 23516 leader_election.cc:288] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 07:16:20.655865 23520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 2 FOLLOWER]: Leader election won for term 2
I20260924 07:16:20.656054 23520 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [term 2 LEADER]: Becoming Leader. State: Replica: 01e79185d5b64ef99f57d815a8bd8ccf, State: Running, Role: LEADER
I20260924 07:16:20.656177 23516 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:16:20.656217 23520 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 38423 } }
I20260924 07:16:20.656605 23520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 38423 } } }
I20260924 07:16:20.656733 23520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:20.656890 23520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e79185d5b64ef99f57d815a8bd8ccf. Latest consensus state: current_term: 2 leader_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e79185d5b64ef99f57d815a8bd8ccf" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 38423 } } }
I20260924 07:16:20.656989 23520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:20.657125 23527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:16:20.657655 23527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:16:20.657811 23527 catalog_manager.cc:1259] Loaded cluster ID: 19b0a6e862704acea25698d8cda35fb3
I20260924 07:16:20.657876 23527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:16:20.659031 23527 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:16:20.659241 23527 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf: Loaded TSK: 0
I20260924 07:16:20.659518 23527 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:16:20.660334 23516 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260924 07:16:20.761488 23519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:20.761832 23519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:20.761898 23519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:20.763576 23519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:20.763677 23519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:20.763749 23519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:16:20.765838 23519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.130.212:54751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:41717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=39911
--tserver_master_addrs=127.10.130.254:38423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:20.766384 23519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:20.766667 23519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:20.772985 23556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:20.773128 23553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:20.773916 23519 server_base.cc:1034] running on GCE node
W20260924 07:16:20.774134 23554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:20.774397 23519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:20.774652 23519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:20.775813 23519 hybrid_clock.cc:648] HybridClock initialized: now 1790234180775791 us; error 33 us; skew 500 ppm
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:20.783069 23519 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:16:20.783645 23519 webserver.cc:460] Webserver started at http://127.10.130.193:39911/ using document root <none> and password file <none>
I20260924 07:16:20.783905 23519 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:20.784004 23519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:16:20.800477 23519 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260924 07:16:20.804534 23564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:20.804855 23519 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260924 07:16:20.804958 23519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "cce9187c8b8e4dfd9a5386fc1a948122"
format_stamp: "Formatted at 2026-09-24 07:16:19 on dist-test-slave-7q7m"
server_key: "275fcaaf64d26fdbe5cd4348fbfda6fc"
server_key_iv: "2ce8081d34fbcbe09258087b7870ac18"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:20.805461 23519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:20.846136 23519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:20.848620 23519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:20.848829 23519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:20.849124 23519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:20.849546 23519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:20.849630 23519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:20.849725 23519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:20.849786 23519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:20.855572 23519 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:41717
I20260924 07:16:20.856099 23519 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:16:20.858914 23677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:41717 every 8 connection(s)
I20260924 07:16:20.860984 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 23519
I20260924 07:16:20.861286 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.130.212:54751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:43583
--local_ip_for_outbound_sockets=127.10.130.194
--tserver_master_addrs=127.10.130.254:38423
--webserver_port=33039
--webserver_interface=127.10.130.194
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.130.212:54751/kms with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:20.874609 23680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:20.859654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:38423 (local address 127.10.130.193:36339)
0924 07:16:20.860172 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:20.860220 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:20.862911 (+ 2691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:20.863642 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:20.863646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:20.863816 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:20.864044 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:20.864051 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:20.865651 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:20.865652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:20.866942 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:20.866946 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:20.868035 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:20.868036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:20.868181 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:20.868787 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:20.868798 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:20.870404 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:20.872686 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:20.872693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:20.872695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:20.873004 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:20.873297 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:20.873298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:20.873298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:20.873443 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:20.873959 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:20.873965 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:20.874120 (+ 155us) client_negotiation.cc:769] Sending connection context
0924 07:16:20.874210 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 07:16:20.874367 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":90,"threads_started":1}
I20260924 07:16:20.875267 23681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:20.860049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:36339 (local address 127.10.130.254:38423)
0924 07:16:20.860243 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:20.860252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:20.860314 (+ 62us) server_negotiation.cc:407] Connection header received
0924 07:16:20.863260 (+ 2946us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:20.863268 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:20.863341 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:20.863483 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:20.864162 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:20.865526 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:20.867045 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:20.867758 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:20.867870 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:20.870537 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:20.870542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:20.870543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:20.870557 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:20.872588 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:20.873100 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:20.873104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:20.873105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:20.873221 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:20.873518 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:20.873519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:20.873519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:20.873846 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:20.873931 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:20.874943 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0924 07:16:20.875067 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":77,"threads_started":1}
I20260924 07:16:20.876135 23678 heartbeater.cc:347] Connected to a master server at 127.10.130.254:38423
I20260924 07:16:20.876230 23678 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:20.876407 23678 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
W20260924 07:16:20.877271 23678 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:38423 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 07:16:20.877334 23678 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:20.877389 23678 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
I20260924 07:16:20.877731 23678 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:20.877796 23678 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
W20260924 07:16:20.878093 23678 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 07:16:20.976847 23682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:20.977195 23682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:20.977288 23682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:20.978963 23682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:20.979071 23682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:20.979157 23682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:16:20.981325 23682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.130.212:54751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:43583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=33039
--tserver_master_addrs=127.10.130.254:38423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:20.981851 23682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:20.982165 23682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:20.989202 23687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:20.989609 23690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:20.990224 23688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:20.990480 23682 server_base.cc:1034] running on GCE node
I20260924 07:16:20.990728 23682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:20.991004 23682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:20.992163 23682 hybrid_clock.cc:648] HybridClock initialized: now 1790234180992139 us; error 37 us; skew 500 ppm
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:20.995409 23682 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:16:20.995966 23682 webserver.cc:460] Webserver started at http://127.10.130.194:33039/ using document root <none> and password file <none>
I20260924 07:16:20.996326 23682 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:20.996424 23682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 07:16:20 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:16:21.016574 23682 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260924 07:16:21.017484 23698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:21.017681 23682 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 07:16:21.017762 23682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "a5fb374b7df1451a845f63eb2c1052ad"
format_stamp: "Formatted at 2026-09-24 07:16:20 on dist-test-slave-7q7m"
server_key: "b71b7a4f926048a6d02f6ebcce574896"
server_key_iv: "27486539d5d2a0e112e5c5f340dae476"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:21.018234 23682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:21.034590 23682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:21.036453 23682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:21.036563 23682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:21.036761 23682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:21.037071 23682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:21.037114 23682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:21.037141 23682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:21.037206 23682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:21.043756 23682 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:43583
I20260924 07:16:21.043846 23811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:43583 every 8 connection(s)
I20260924 07:16:21.044157 23682 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
Sep 24 07:16:21 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234180, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:21.053896 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 23682
I20260924 07:16:21.054193 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.130.212:54751/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:46045
--local_ip_for_outbound_sockets=127.10.130.195
--tserver_master_addrs=127.10.130.254:38423
--webserver_port=41079
--webserver_interface=127.10.130.195
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.130.212:54751/kms with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:16:21.055321 23681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:21.045126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:38471 (local address 127.10.130.254:38423)
0924 07:16:21.045176 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:21.045178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:21.045188 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:16:21.045713 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:21.045714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:21.045732 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:21.045811 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:21.046481 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:21.047254 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:21.048738 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:21.049045 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:21.049123 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:21.051712 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:21.051713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:21.051714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:21.051721 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:21.053452 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:21.054251 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:21.054253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:21.054253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:21.054335 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:21.054615 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:21.054616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:21.054617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:21.054743 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:21.054781 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:21.055138 (+ 357us) server_negotiation.cc:299] Negotiation successful
0924 07:16:21.055225 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:16:21.055827 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:21.044901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:38423 (local address 127.10.130.194:38471)
0924 07:16:21.045048 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:21.045089 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:21.045573 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:21.045886 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:21.045890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:21.046102 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:21.046369 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:21.046376 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:21.047353 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:21.047355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:21.048649 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:21.048652 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:21.049148 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:21.049150 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:21.049287 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:21.049856 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:21.049865 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:21.051554 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:21.053571 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:21.053579 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:21.053582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:21.054114 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:21.054408 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:21.054409 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:21.054410 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:21.054547 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:21.054822 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:21.054826 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:21.054999 (+ 173us) client_negotiation.cc:769] Sending connection context
0924 07:16:21.055437 (+ 438us) client_negotiation.cc:240] Negotiation successful
0924 07:16:21.055598 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
I20260924 07:16:21.060076 23812 heartbeater.cc:347] Connected to a master server at 127.10.130.254:38423
I20260924 07:16:21.060248 23812 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:21.060503 23812 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
W20260924 07:16:21.061220 23812 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:38423 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 07:16:21.061347 23812 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:21.061479 23812 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
I20260924 07:16:21.061817 23812 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:21.061945 23812 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
W20260924 07:16:21.062260 23812 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 07:16:21.149887 23815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:21.150164 23815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:21.150197 23815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:21.151820 23815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:21.151926 23815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:21.152019 23815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:16:21.154039 23815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.130.212:54751/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:46045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=41079
--tserver_master_addrs=127.10.130.254:38423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:21.154392 23815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:21.154656 23815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:21.161103 23820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:21.161115 23821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:21.161669 23815 server_base.cc:1034] running on GCE node
W20260924 07:16:21.168031 23823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:21.168285 23815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:21.168587 23815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:21.169763 23815 hybrid_clock.cc:648] HybridClock initialized: now 1790234181169745 us; error 36 us; skew 500 ppm
Sep 24 07:16:21 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234181, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:21.173043 23815 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:16:21.173594 23815 webserver.cc:460] Webserver started at http://127.10.130.195:41079/ using document root <none> and password file <none>
I20260924 07:16:21.173836 23815 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:21.173925 23815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 07:16:21 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234181, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:16:21.188925 23815 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260924 07:16:21.189925 23831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:21.190145 23815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 07:16:21.190266 23815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "5158fffa2db5426c8db6ceb3774e6205"
format_stamp: "Formatted at 2026-09-24 07:16:20 on dist-test-slave-7q7m"
server_key: "e585e268c9c50577f460bc4f7bd8c9f5"
server_key_iv: "653aa10a0b10842c4feec61479002e71"
server_key_version: "kuduclusterkey@0"
I20260924 07:16:21.190785 23815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:21.203815 23815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:21.205574 23815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:21.205690 23815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:21.205873 23815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:21.206204 23815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:21.206243 23815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:21.206271 23815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:21.206292 23815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:21.212852 23815 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:46045
I20260924 07:16:21.213017 23945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:46045 every 8 connection(s)
I20260924 07:16:21.213384 23815 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:16:21.220000 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 23815
Sep 24 07:16:21 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234181, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:21.224117 23681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:21.214706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:56315 (local address 127.10.130.254:38423)
0924 07:16:21.214777 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:21.214780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:21.214986 (+ 206us) server_negotiation.cc:407] Connection header received
0924 07:16:21.215628 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:21.215629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:21.215642 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:21.215714 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:21.216460 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:21.217213 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:21.218420 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:21.218719 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:21.218791 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:21.220895 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:21.220896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:21.220897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:21.220904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:21.222573 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:21.223041 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:21.223042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:21.223043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:21.223106 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:21.223387 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:21.223387 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:21.223388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:21.223518 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:21.223587 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:21.223888 (+ 301us) server_negotiation.cc:299] Negotiation successful
0924 07:16:21.224025 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 07:16:21.224329 23948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:21.214420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:38423 (local address 127.10.130.195:56315)
0924 07:16:21.214846 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:21.214894 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:21.215507 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:21.215791 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:21.215796 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:21.216073 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:21.216353 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:21.216360 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:21.217327 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:21.217328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:21.218329 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:21.218331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:21.218806 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:21.218807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:21.218895 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:21.219354 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:21.219362 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:21.220779 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:21.222658 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:21.222664 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:21.222667 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:21.222940 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:21.223180 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:21.223181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:21.223181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:21.223320 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:21.223587 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:21.223592 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:21.223734 (+ 142us) client_negotiation.cc:769] Sending connection context
0924 07:16:21.223850 (+ 116us) client_negotiation.cc:240] Negotiation successful
0924 07:16:21.224022 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":82,"threads_started":1}
I20260924 07:16:21.224884 23946 heartbeater.cc:347] Connected to a master server at 127.10.130.254:38423
I20260924 07:16:21.224982 23946 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:21.225163 23946 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
W20260924 07:16:21.225823 23946 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:38423 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 07:16:21.225891 23946 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:21.225970 23946 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
I20260924 07:16:21.226336 23946 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:21.226403 23946 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
W20260924 07:16:21.226722 23946 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 07:16:21.231729 23950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:21.220541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39748 (local address 127.10.130.254:38423)
0924 07:16:21.220676 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:21.220679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:21.223190 (+ 2511us) server_negotiation.cc:407] Connection header received
0924 07:16:21.223206 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:21.223206 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:21.223217 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:21.223268 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:21.226799 (+ 3531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:21.227276 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:21.228104 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:21.228317 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:21.228368 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:21.229366 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:21.229368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:21.229368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:21.229374 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:21.230756 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:21.231046 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:21.231047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:21.231047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:21.231101 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:21.231255 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:21.231256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:21.231257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:21.231370 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:21.231524 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:21.231560 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 07:16:21.231634 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
Sep 24 07:16:21 dist-test-slave-7q7m krb5kdc[22484](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790234181, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:21.878281 23678 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:21.878436 23678 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
I20260924 07:16:21.878888 23480 ts_manager.cc:194] Registered new tserver with Master: cce9187c8b8e4dfd9a5386fc1a948122 (127.10.130.193:41717)
I20260924 07:16:21.879935 23480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:36339
I20260924 07:16:22.062469 23812 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:22.062613 23812 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
I20260924 07:16:22.062883 23480 ts_manager.cc:194] Registered new tserver with Master: a5fb374b7df1451a845f63eb2c1052ad (127.10.130.194:43583)
I20260924 07:16:22.063282 23480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:38471
Sep 24 07:16:22 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234181, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:16:22.226852 23946 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:22.227006 23946 heartbeater.cc:511] Master 127.10.130.254:38423 requested a full tablet report, sending...
I20260924 07:16:22.227408 23480 ts_manager.cc:194] Registered new tserver with Master: 5158fffa2db5426c8db6ceb3774e6205 (127.10.130.195:46045)
I20260924 07:16:22.227944 23480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:56315
I20260924 07:16:22.228863 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:16:22.238401 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:22.230473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39758 (local address 127.10.130.254:38423)
0924 07:16:22.230743 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:22.230745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:22.230851 (+ 106us) server_negotiation.cc:407] Connection header received
0924 07:16:22.230951 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:22.230952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:22.230964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:22.231007 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:22.231716 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.232531 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.233540 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.233852 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.233897 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:22.235375 (+ 1478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:22.235376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:22.235376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:22.235395 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:22.237166 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:22.237566 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:22.237567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:22.237568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:22.237629 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:22.237818 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:22.237819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:22.237819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:22.237937 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:22.238009 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:22.238208 (+ 199us) server_negotiation.cc:299] Negotiation successful
0924 07:16:22.238289 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":70,"threads_started":1}
I20260924 07:16:22.240878 23479 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:16:22.243189 23479 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:16:22.258663 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:22.246521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:43583 (local address 127.0.0.1:33118)
0924 07:16:22.252088 (+ 5567us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:22.252137 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:22.252205 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:22.252810 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:22.252814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:22.252816 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:22.252898 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:22.252900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.254271 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.254272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:22.255849 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:22.255851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.257944 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.257945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:22.258030 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:22.258043 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 07:16:22.258104 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 07:16:22.258573 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5545,"mutex_wait_us":408,"thread_start_us":63,"threads_started":1}
I20260924 07:16:22.259217 23976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:22.246927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33118 (local address 127.10.130.194:43583)
0924 07:16:22.252607 (+ 5680us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:22.252611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:22.252620 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:16:22.252651 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:22.252653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:22.252668 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:22.252729 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:22.253089 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.254183 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.255950 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.256755 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.256821 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:22.256876 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:22.259074 (+ 2198us) server_negotiation.cc:299] Negotiation successful
0924 07:16:22.259114 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5646,"thread_start_us":54,"threads_started":1}
I20260924 07:16:22.261236 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:22.246302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:46045 (local address 127.0.0.1:56520)
0924 07:16:22.252081 (+ 5779us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:22.252136 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:22.252205 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:22.256260 (+ 4055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:22.256261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:22.256263 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:22.256341 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:22.256343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.259200 (+ 2857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.259201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:22.260110 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:22.260113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.261021 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.261022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:22.261088 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:22.261092 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:16:22.261125 (+ 33us) client_negotiation.cc:240] Negotiation successful
0924 07:16:22.261158 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5736,"thread_start_us":69,"threads_started":1}
I20260924 07:16:22.261741 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:22.247117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:41717 (local address 127.0.0.1:34898)
0924 07:16:22.252985 (+ 5868us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:22.252993 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:22.253031 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:22.256643 (+ 3612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:22.256644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:22.256645 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:22.256687 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:22.256689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.257991 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.257992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:22.259115 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:22.259118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.261545 (+ 2427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.261546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:22.261603 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:22.261607 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:16:22.261648 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 07:16:22.261683 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5853,"thread_start_us":63,"threads_started":1}
I20260924 07:16:22.262068 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:22.247399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34898 (local address 127.10.130.193:41717)
0924 07:16:22.252348 (+ 4949us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:22.252353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:22.256431 (+ 4078us) server_negotiation.cc:407] Connection header received
0924 07:16:22.256473 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:22.256476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:22.256493 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:22.256563 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:22.256786 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.257894 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.260069 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.260809 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.260878 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:22.260937 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:22.261940 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0924 07:16:22.261983 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4909,"thread_start_us":55,"threads_started":1}
I20260924 07:16:22.262363 23975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:22.246781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56520 (local address 127.10.130.195:46045)
0924 07:16:22.252337 (+ 5556us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:22.252342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:22.252352 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:16:22.252386 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:22.252389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:22.252404 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:22.252471 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:22.256928 (+ 4457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.258458 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.260200 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.260873 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.260939 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:22.260989 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:22.262253 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0924 07:16:22.262296 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5520,"thread_start_us":53,"threads_started":1}
I20260924 07:16:22.263231 23612 tablet_service.cc:1467] Processing CreateTablet for tablet 46c630eab1d74cbabfe1f4f8d1d48292 (DEFAULT_TABLE table=test-table [id=4cafb5a15cbe4a25a5775812c64868d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:16:22.263594 23612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46c630eab1d74cbabfe1f4f8d1d48292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:22.268075 23979 tablet_bootstrap.cc:492] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122: Bootstrap starting.
I20260924 07:16:22.268328 23880 tablet_service.cc:1467] Processing CreateTablet for tablet 46c630eab1d74cbabfe1f4f8d1d48292 (DEFAULT_TABLE table=test-table [id=4cafb5a15cbe4a25a5775812c64868d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:16:22.268693 23880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46c630eab1d74cbabfe1f4f8d1d48292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:22.268775 23979 tablet_bootstrap.cc:654] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:22.269058 23979 log.cc:824] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:22.269763 23979 tablet_bootstrap.cc:492] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122: No bootstrap required, opened a new log
I20260924 07:16:22.269822 23979 ts_tablet_manager.cc:1397] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:16:22.270175 23746 tablet_service.cc:1467] Processing CreateTablet for tablet 46c630eab1d74cbabfe1f4f8d1d48292 (DEFAULT_TABLE table=test-table [id=4cafb5a15cbe4a25a5775812c64868d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:16:22.270511 23746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46c630eab1d74cbabfe1f4f8d1d48292. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:22.272251 23979 raft_consensus.cc:348] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5158fffa2db5426c8db6ceb3774e6205" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 46045 } } peers { permanent_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41717 } } peers { permanent_uuid: "a5fb374b7df1451a845f63eb2c1052ad" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43583 } }
I20260924 07:16:22.272400 23979 raft_consensus.cc:374] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:22.272428 23979 raft_consensus.cc:729] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cce9187c8b8e4dfd9a5386fc1a948122, State: Initialized, Role: FOLLOWER
I20260924 07:16:22.272562 23979 consensus_queue.cc:260] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5158fffa2db5426c8db6ceb3774e6205" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 46045 } } peers { permanent_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41717 } } peers { permanent_uuid: "a5fb374b7df1451a845f63eb2c1052ad" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43583 } }
I20260924 07:16:22.272809 23979 ts_tablet_manager.cc:1428] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:16:22.272913 23678 heartbeater.cc:503] Master 127.10.130.254:38423 was elected leader, sending a full tablet report...
I20260924 07:16:22.276316 23983 tablet_bootstrap.cc:492] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205: Bootstrap starting.
I20260924 07:16:22.277069 23983 tablet_bootstrap.cc:654] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:22.277547 23983 log.cc:824] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:22.278308 23983 tablet_bootstrap.cc:492] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205: No bootstrap required, opened a new log
I20260924 07:16:22.278373 23983 ts_tablet_manager.cc:1397] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:16:22.278362 23982 tablet_bootstrap.cc:492] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad: Bootstrap starting.
I20260924 07:16:22.278923 23982 tablet_bootstrap.cc:654] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:22.279172 23982 log.cc:824] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:22.279892 23982 tablet_bootstrap.cc:492] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad: No bootstrap required, opened a new log
I20260924 07:16:22.279947 23982 ts_tablet_manager.cc:1397] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:16:22.280329 23983 raft_consensus.cc:348] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5158fffa2db5426c8db6ceb3774e6205" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 46045 } } peers { permanent_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41717 } } peers { permanent_uuid: "a5fb374b7df1451a845f63eb2c1052ad" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43583 } }
I20260924 07:16:22.280491 23983 raft_consensus.cc:374] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:22.280526 23983 raft_consensus.cc:729] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5158fffa2db5426c8db6ceb3774e6205, State: Initialized, Role: FOLLOWER
I20260924 07:16:22.280627 23983 consensus_queue.cc:260] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5158fffa2db5426c8db6ceb3774e6205" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 46045 } } peers { permanent_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41717 } } peers { permanent_uuid: "a5fb374b7df1451a845f63eb2c1052ad" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43583 } }
I20260924 07:16:22.280846 23983 ts_tablet_manager.cc:1428] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:16:22.281028 23946 heartbeater.cc:503] Master 127.10.130.254:38423 was elected leader, sending a full tablet report...
I20260924 07:16:22.282246 23982 raft_consensus.cc:348] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5158fffa2db5426c8db6ceb3774e6205" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 46045 } } peers { permanent_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41717 } } peers { permanent_uuid: "a5fb374b7df1451a845f63eb2c1052ad" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43583 } }
I20260924 07:16:22.282390 23982 raft_consensus.cc:374] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:22.282418 23982 raft_consensus.cc:729] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5fb374b7df1451a845f63eb2c1052ad, State: Initialized, Role: FOLLOWER
I20260924 07:16:22.282517 23982 consensus_queue.cc:260] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5158fffa2db5426c8db6ceb3774e6205" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 46045 } } peers { permanent_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41717 } } peers { permanent_uuid: "a5fb374b7df1451a845f63eb2c1052ad" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43583 } }
I20260924 07:16:22.282753 23982 ts_tablet_manager.cc:1428] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:16:22.282950 23812 heartbeater.cc:503] Master 127.10.130.254:38423 was elected leader, sending a full tablet report...
W20260924 07:16:22.295163 23813 tablet.cc:2369] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 07:16:22.360993 23679 tablet.cc:2369] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 07:16:22.464740 23947 tablet.cc:2369] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 07:16:22 dist-test-slave-7q7m krb5kdc[22484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234181, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for HTTP/127.10.130.212@KRBTEST.COM
I20260924 07:16:22.490173 23981 raft_consensus.cc:482] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:16:22.490316 23981 raft_consensus.cc:504] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5158fffa2db5426c8db6ceb3774e6205" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 46045 } } peers { permanent_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41717 } } peers { permanent_uuid: "a5fb374b7df1451a845f63eb2c1052ad" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43583 } }
I20260924 07:16:22.490763 23981 leader_election.cc:288] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5158fffa2db5426c8db6ceb3774e6205 (127.10.130.195:46045), a5fb374b7df1451a845f63eb2c1052ad (127.10.130.194:43583)
I20260924 07:16:22.495244 23975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:22.491099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:48997 (local address 127.10.130.195:46045)
0924 07:16:22.491349 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:22.491352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:22.491362 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:16:22.491393 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:22.491394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:22.491405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:22.491471 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:22.491641 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.492346 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.494217 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.494712 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.494753 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:22.494761 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:22.495137 (+ 376us) server_negotiation.cc:299] Negotiation successful
0924 07:16:22.495179 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206}
I20260924 07:16:22.495829 23989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:22.490898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:46045 (local address 127.10.130.193:48997)
0924 07:16:22.491166 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:22.491176 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:22.491220 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:22.491486 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:22.491487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:22.491488 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:22.491563 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:22.491565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.492665 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.492666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:22.494096 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:22.494098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.494993 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.494994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:22.495067 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:22.495070 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:16:22.495692 (+ 622us) client_negotiation.cc:240] Negotiation successful
0924 07:16:22.495740 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":27,"thread_start_us":96,"threads_started":1}
I20260924 07:16:22.496452 23886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46c630eab1d74cbabfe1f4f8d1d48292" candidate_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5158fffa2db5426c8db6ceb3774e6205" is_pre_election: true
I20260924 07:16:22.496634 23886 raft_consensus.cc:2454] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cce9187c8b8e4dfd9a5386fc1a948122 in term 0.
I20260924 07:16:22.496867 23568 leader_election.cc:302] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5158fffa2db5426c8db6ceb3774e6205, cce9187c8b8e4dfd9a5386fc1a948122; no voters:
I20260924 07:16:22.497002 23981 raft_consensus.cc:2781] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:16:22.497084 23981 raft_consensus.cc:482] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:16:22.497112 23981 raft_consensus.cc:3037] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:22.498076 23981 raft_consensus.cc:504] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5158fffa2db5426c8db6ceb3774e6205" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 46045 } } peers { permanent_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41717 } } peers { permanent_uuid: "a5fb374b7df1451a845f63eb2c1052ad" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43583 } }
I20260924 07:16:22.498229 23981 leader_election.cc:288] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [CANDIDATE]: Term 1 election: Requested vote from peers 5158fffa2db5426c8db6ceb3774e6205 (127.10.130.195:46045), a5fb374b7df1451a845f63eb2c1052ad (127.10.130.194:43583)
I20260924 07:16:22.498432 23886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46c630eab1d74cbabfe1f4f8d1d48292" candidate_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5158fffa2db5426c8db6ceb3774e6205"
I20260924 07:16:22.498507 23886 raft_consensus.cc:3037] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:22.499298 23886 raft_consensus.cc:2454] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cce9187c8b8e4dfd9a5386fc1a948122 in term 1.
I20260924 07:16:22.499511 23568 leader_election.cc:302] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5158fffa2db5426c8db6ceb3774e6205, cce9187c8b8e4dfd9a5386fc1a948122; no voters:
I20260924 07:16:22.499612 23981 raft_consensus.cc:2781] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:16:22.499779 23981 raft_consensus.cc:686] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [term 1 LEADER]: Becoming Leader. State: Replica: cce9187c8b8e4dfd9a5386fc1a948122, State: Running, Role: LEADER
I20260924 07:16:22.499883 23981 consensus_queue.cc:237] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5158fffa2db5426c8db6ceb3774e6205" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 46045 } } peers { permanent_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41717 } } peers { permanent_uuid: "a5fb374b7df1451a845f63eb2c1052ad" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43583 } }
I20260924 07:16:22.500934 23976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:22.491549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:41041 (local address 127.10.130.194:43583)
0924 07:16:22.491604 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:22.491606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:22.491613 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:16:22.491638 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:22.491639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:22.491647 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:22.491694 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:22.492173 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.492797 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.500087 (+ 7290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.500604 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.500647 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:22.500655 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:22.500840 (+ 185us) server_negotiation.cc:299] Negotiation successful
0924 07:16:22.500875 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 07:16:22.500882 23480 catalog_manager.cc:5495] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 reported cstate change: term changed from 0 to 1, leader changed from <none> to cce9187c8b8e4dfd9a5386fc1a948122 (127.10.130.193). New cstate: current_term: 1 leader_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5158fffa2db5426c8db6ceb3774e6205" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 46045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 41717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5fb374b7df1451a845f63eb2c1052ad" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43583 } health_report { overall_health: UNKNOWN } } }
I20260924 07:16:22.501329 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:22.491032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:43583 (local address 127.10.130.193:41041)
0924 07:16:22.491284 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:22.491290 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:22.491318 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:22.492023 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:22.492024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:22.492026 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:22.492091 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:22.492093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.492878 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.492878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:22.494096 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:22.494098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.500707 (+ 6609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.500707 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:22.500773 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:22.500776 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:16:22.501181 (+ 405us) client_negotiation.cc:240] Negotiation successful
0924 07:16:22.501223 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":19,"thread_start_us":95,"threads_started":1}
I20260924 07:16:22.501777 23766 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46c630eab1d74cbabfe1f4f8d1d48292" candidate_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5fb374b7df1451a845f63eb2c1052ad" is_pre_election: true
I20260924 07:16:22.501951 23766 raft_consensus.cc:2454] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cce9187c8b8e4dfd9a5386fc1a948122 in term 0.
I20260924 07:16:22.502095 23765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "46c630eab1d74cbabfe1f4f8d1d48292" candidate_uuid: "cce9187c8b8e4dfd9a5386fc1a948122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5fb374b7df1451a845f63eb2c1052ad"
I20260924 07:16:22.502167 23765 raft_consensus.cc:3037] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:22.502971 23765 raft_consensus.cc:2454] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cce9187c8b8e4dfd9a5386fc1a948122 in term 1.
I20260924 07:16:22.533957 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:22.530664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34914 (local address 127.10.130.193:41717)
0924 07:16:22.530705 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:22.530707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:22.530713 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:16:22.530753 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:22.530754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:22.530775 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:22.530830 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:16:22.530991 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.531711 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.532831 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:22.533146 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:22.533199 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:22.533394 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:16:22.533527 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:16:22.533601 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:22.533852 (+ 251us) server_negotiation.cc:299] Negotiation successful
0924 07:16:22.533878 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:16:22.536208 23765 raft_consensus.cc:1261] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad [term 1 FOLLOWER]: Refusing update from remote peer cce9187c8b8e4dfd9a5386fc1a948122: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:16:22.536574 23981 consensus_queue.cc:1035] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5fb374b7df1451a845f63eb2c1052ad" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 43583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:16:22.537654 23886 raft_consensus.cc:1261] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205 [term 1 FOLLOWER]: Refusing update from remote peer cce9187c8b8e4dfd9a5386fc1a948122: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:16:22.538033 23981 consensus_queue.cc:1035] T 46c630eab1d74cbabfe1f4f8d1d48292 P cce9187c8b8e4dfd9a5386fc1a948122 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5158fffa2db5426c8db6ceb3774e6205" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 46045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:16:22.541395 23998 mvcc.cc:205] Tried to move back new op lower bound from 7332799211665383424 to 7332799211520266240. Current Snapshot: MvccSnapshot[applied={T|T < 7332799211665383424}]
I20260924 07:16:22.545054 23995 mvcc.cc:205] Tried to move back new op lower bound from 7332799211665383424 to 7332799211520266240. Current Snapshot: MvccSnapshot[applied={T|T < 7332799211665383424}]
I20260924 07:16:22.545514 23996 mvcc.cc:205] Tried to move back new op lower bound from 7332799211665383424 to 7332799211520266240. Current Snapshot: MvccSnapshot[applied={T|T < 7332799211665383424}]
I20260924 07:16:22.549573 23480 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:16:22.549676 23480 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:16:22.550557 23480 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01e79185d5b64ef99f57d815a8bd8ccf: Sending DeleteTablet for 3 replicas of tablet 46c630eab1d74cbabfe1f4f8d1d48292
I20260924 07:16:22.550977 23880 tablet_service.cc:1514] Processing DeleteTablet for tablet 46c630eab1d74cbabfe1f4f8d1d48292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:22 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:56520
I20260924 07:16:22.551043 23746 tablet_service.cc:1514] Processing DeleteTablet for tablet 46c630eab1d74cbabfe1f4f8d1d48292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:22 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:33118
I20260924 07:16:22.551437 23612 tablet_service.cc:1514] Processing DeleteTablet for tablet 46c630eab1d74cbabfe1f4f8d1d48292 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:22 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:34898
I20260924 07:16:22.552078 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 23519
I20260924 07:16:22.558981 24005 tablet_replica.cc:330] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad: stopping tablet replica
I20260924 07:16:22.559157 24005 raft_consensus.cc:2229] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:16:22.559376 24005 raft_consensus.cc:2258] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:16:22.561717 24004 tablet_replica.cc:330] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205: stopping tablet replica
I20260924 07:16:22.561914 24004 raft_consensus.cc:2229] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:16:22.563915 24005 ts_tablet_manager.cc:1905] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:16:22.566937 24005 ts_tablet_manager.cc:1918] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:16:22.567070 24005 log.cc:1196] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/46c630eab1d74cbabfe1f4f8d1d48292
I20260924 07:16:22.567415 24005 ts_tablet_manager.cc:1939] T 46c630eab1d74cbabfe1f4f8d1d48292 P a5fb374b7df1451a845f63eb2c1052ad: Deleting consensus metadata
I20260924 07:16:22.568953 23466 catalog_manager.cc:4841] TS a5fb374b7df1451a845f63eb2c1052ad (127.10.130.194:43583): tablet 46c630eab1d74cbabfe1f4f8d1d48292 (table test-table [id=4cafb5a15cbe4a25a5775812c64868d8]) successfully deleted
W20260924 07:16:22.571696 23465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:41717 (error 108)
I20260924 07:16:22.572402 24004 raft_consensus.cc:2258] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:16:22.572963 24004 ts_tablet_manager.cc:1905] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:16:22.574600 24004 ts_tablet_manager.cc:1918] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:16:22.574713 24004 log.cc:1196] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/46c630eab1d74cbabfe1f4f8d1d48292
I20260924 07:16:22.575002 24004 ts_tablet_manager.cc:1939] T 46c630eab1d74cbabfe1f4f8d1d48292 P 5158fffa2db5426c8db6ceb3774e6205: Deleting consensus metadata
I20260924 07:16:22.575673 23467 catalog_manager.cc:4841] TS 5158fffa2db5426c8db6ceb3774e6205 (127.10.130.195:46045): tablet 46c630eab1d74cbabfe1f4f8d1d48292 (table test-table [id=4cafb5a15cbe4a25a5775812c64868d8]) successfully deleted
I20260924 07:16:22.575889 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 23682
I20260924 07:16:22.576938 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:22.575936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:41717 (local address 127.0.0.1:34918)
0924 07:16:22.576851 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:22.576900 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:41717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":890}
W20260924 07:16:22.577513 23465 catalog_manager.cc:4584] TS cce9187c8b8e4dfd9a5386fc1a948122 (127.10.130.193:41717): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 46c630eab1d74cbabfe1f4f8d1d48292: Network error: Client connection negotiation failed: client connection to 127.10.130.193:41717: connect: Connection refused (error 111)
I20260924 07:16:22.583716 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 23815
I20260924 07:16:22.590188 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 23447
I20260924 07:16:22.595162 10763 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 07:16:22.938541 10763 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 07:16:22.938624 10763 mini_ranger.cc:66] Stopping Ranger...
I20260924 07:16:23.271541 10763 mini_ranger.cc:68] Stopped Ranger
2026-09-24 07:16:23.271 UTC [22514] LOG: received smart shutdown request
2026-09-24 07:16:23.275 UTC [22514] LOG: background worker "logical replication launcher" (PID 22523) exited with exit code 1
2026-09-24 07:16:23.275 UTC [22518] LOG: shutting down
2026-09-24 07:16:23.413 UTC [22514] LOG: database system is shut down
2026-09-24T07:16:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (44585 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:16:23 dist-test-slave-7q7m krb5kdc[24016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:16:23 dist-test-slave-7q7m krb5kdc[24016](info): set up 2 sockets
Sep 24 07:16:23 dist-test-slave-7q7m krb5kdc[24016](info): commencing operation
krb5kdc: starting...
W20260924 07:16:24.455595 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.933s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:16:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:16:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:16:24.560899 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:44777
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:44777,127.10.130.253:41597,127.10.130.252:40675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:24.637364 24029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:24.637563 24029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:24.637591 24029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:24.638691 24029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:16:24.638732 24029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:24.638751 24029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:24.638767 24029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:16:24.638782 24029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:16:24.640226 24029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:44777,127.10.130.253:41597,127.10.130.252:40675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:44777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:24.640468 24029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:24.640671 24029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:24.645150 24034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:24.645192 24037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:24.645192 24035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:24.645419 24029 server_base.cc:1034] running on GCE node
I20260924 07:16:24.645660 24029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:24.645924 24029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:24.647074 24029 hybrid_clock.cc:648] HybridClock initialized: now 1790234184647060 us; error 28 us; skew 500 ppm
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:24.652036 24029 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:16:24.652434 24029 webserver.cc:460] Webserver started at http://127.10.130.254:44275/ using document root <none> and password file <none>
I20260924 07:16:24.652634 24029 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:24.652680 24029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:24.652796 24029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:16:24.653761 24029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "1f07390429354e1da1aa6f70a2387ba1"
format_stamp: "Formatted at 2026-09-24 07:16:24 on dist-test-slave-7q7m"
server_key: "0140c638bd3016fc878eda3b811c695d"
server_key_iv: "c7027aeee67f6b98b6d70f81b98ed397"
server_key_version: "encryptionkey@0"
I20260924 07:16:24.654076 24029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "1f07390429354e1da1aa6f70a2387ba1"
format_stamp: "Formatted at 2026-09-24 07:16:24 on dist-test-slave-7q7m"
server_key: "0140c638bd3016fc878eda3b811c695d"
server_key_iv: "c7027aeee67f6b98b6d70f81b98ed397"
server_key_version: "encryptionkey@0"
I20260924 07:16:24.655232 24029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:16:24.655835 24044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:24.655972 24029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:16:24.656075 24029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "1f07390429354e1da1aa6f70a2387ba1"
format_stamp: "Formatted at 2026-09-24 07:16:24 on dist-test-slave-7q7m"
server_key: "0140c638bd3016fc878eda3b811c695d"
server_key_iv: "c7027aeee67f6b98b6d70f81b98ed397"
server_key_version: "encryptionkey@0"
I20260924 07:16:24.656137 24029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:24.666186 24029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:24.667696 24029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:24.667814 24029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:24.671386 24029 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:44777
I20260924 07:16:24.671416 24096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:44777 every 8 connection(s)
I20260924 07:16:24.671708 24029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:16:24.672519 24097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:24.673949 24097 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:16:24.674458 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 24029
I20260924 07:16:24.674559 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:16:24.674746 10763 external_mini_cluster.cc:1428] Setting key 2b6aec12971a3cd6ada4f011ab364377
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.130.253@KRBTEST.COM; defaulting to no policy
I20260924 07:16:24.683862 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.674618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52562 (local address 127.10.130.254:44777)
0924 07:16:24.674791 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.674795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.674875 (+ 80us) server_negotiation.cc:407] Connection header received
0924 07:16:24.675519 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.675522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.675544 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.675627 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.676318 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.677445 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.678625 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.678925 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.679003 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.681180 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.681186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.681187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.681192 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.682569 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.682948 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.682950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.682950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.683033 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.683192 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.683192 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.683193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.683384 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.683441 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.683560 (+ 119us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.683680 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":53,"threads_started":1}
I20260924 07:16:24.683876 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.674568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:44777 (local address 127.0.0.1:52562)
0924 07:16:24.674736 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.674780 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:24.675359 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:24.675665 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:24.675668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:24.676003 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:24.676228 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.676234 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.677529 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.677530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.678556 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.678559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.679007 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.679008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.679090 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.679545 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:24.679556 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:24.681103 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:24.682637 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.682643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.682644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.682886 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.683081 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.683082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.683082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.683151 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.683429 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:24.683432 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:24.683478 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 07:16:24.683566 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 07:16:24.683742 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":74,"thread_start_us":77,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.253@KRBTEST.COM" created.
I20260924 07:16:24.684587 24097 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:16:24.684890 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.684792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:41597 (local address 127.0.0.1:60924)
0924 07:16:24.684822 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.684871 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260924 07:16:24.685010 24046 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.130.253:41597: connect: Connection refused (error 111)
I20260924 07:16:24.685408 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.685308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:41597 (local address 127.0.0.1:60926)
0924 07:16:24.685357 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.685386 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 07:16:24.685498 24097 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.253:41597: Network error: Client connection negotiation failed: client connection to 127.10.130.253:41597: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.253.keytab.
Entry for principal kudu/127.10.130.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.253.keytab.
I20260924 07:16:24.696010 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.253:41597
--webserver_interface=127.10.130.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.253.keytab
--principal=kudu/127.10.130.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:44777,127.10.130.253:41597,127.10.130.252:40675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:16:24.734709 24097 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } attempt: 1
I20260924 07:16:24.735169 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.735050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:41597 (local address 127.0.0.1:60936)
0924 07:16:24.735108 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.735139 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 07:16:24.735464 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.735387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:41597 (local address 127.0.0.1:60938)
0924 07:16:24.735422 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.735445 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260924 07:16:24.735548 24097 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.253:41597: Network error: Client connection negotiation failed: client connection to 127.10.130.253:41597: connect: Connection refused (error 111)
W20260924 07:16:24.773834 24104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:24.774039 24104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:24.774067 24104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:24.775400 24104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:16:24.775454 24104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:24.775475 24104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:24.775491 24104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:16:24.775516 24104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:16:24.777519 24104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:44777,127.10.130.253:41597,127.10.130.252:40675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.253.keytab
--principal=kudu/127.10.130.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.253:41597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.130.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:24.777699 24104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:24.777860 24104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:24.782080 24112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:24.782116 24110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:24.782143 24109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:24.782196 24104 server_base.cc:1034] running on GCE node
I20260924 07:16:24.782548 24104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:24.782804 24104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:24.783931 24104 hybrid_clock.cc:648] HybridClock initialized: now 1790234184783920 us; error 24 us; skew 500 ppm
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:24.786180 24104 init.cc:377] Logged in from keytab as kudu/127.10.130.253@KRBTEST.COM (short username kudu)
I20260924 07:16:24.786583 24104 webserver.cc:460] Webserver started at http://127.10.130.253:43677/ using document root <none> and password file <none>
I20260924 07:16:24.786763 24104 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:24.786809 24104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:24.786927 24104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:16:24.787885 24104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/data/instance:
uuid: "3e1e258075b64a918ac2e12b11456687"
format_stamp: "Formatted at 2026-09-24 07:16:24 on dist-test-slave-7q7m"
server_key: "52f1b101b649c325e1a1e1b584fbbe0d"
server_key_iv: "1f1cf74d769a3eb5f353ae7189b9a10e"
server_key_version: "encryptionkey@0"
I20260924 07:16:24.788255 24104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/wal/instance:
uuid: "3e1e258075b64a918ac2e12b11456687"
format_stamp: "Formatted at 2026-09-24 07:16:24 on dist-test-slave-7q7m"
server_key: "52f1b101b649c325e1a1e1b584fbbe0d"
server_key_iv: "1f1cf74d769a3eb5f353ae7189b9a10e"
server_key_version: "encryptionkey@0"
I20260924 07:16:24.789418 24104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:16:24.790117 24119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:24.790273 24104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:16:24.790351 24104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/wal
uuid: "3e1e258075b64a918ac2e12b11456687"
format_stamp: "Formatted at 2026-09-24 07:16:24 on dist-test-slave-7q7m"
server_key: "52f1b101b649c325e1a1e1b584fbbe0d"
server_key_iv: "1f1cf74d769a3eb5f353ae7189b9a10e"
server_key_version: "encryptionkey@0"
I20260924 07:16:24.790416 24104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:24.799852 24104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:24.801640 24104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:24.801765 24104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:24.803686 24097 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } attempt: 2
I20260924 07:16:24.804112 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.803998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:41597 (local address 127.0.0.1:60944)
0924 07:16:24.804048 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.804082 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260924 07:16:24.804471 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.804377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:41597 (local address 127.0.0.1:60954)
0924 07:16:24.804418 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.804443 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:41597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260924 07:16:24.804553 24097 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.253:41597: Network error: Client connection negotiation failed: client connection to 127.10.130.253:41597: connect: Connection refused (error 111)
I20260924 07:16:24.805727 24104 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.253:41597
I20260924 07:16:24.805757 24171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.253:41597 every 8 connection(s)
I20260924 07:16:24.806071 24104 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/data/info.pb
I20260924 07:16:24.806834 24172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:24.808312 24172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:16:24.810016 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 24104
I20260924 07:16:24.810096 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-1/wal/instance
I20260924 07:16:24.810240 10763 external_mini_cluster.cc:1428] Setting key 78db9b2b9c63e90fcb8bcb9faed19427
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.253@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:24.816365 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.808900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52566 (local address 127.10.130.254:44777)
0924 07:16:24.808945 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.808947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.809134 (+ 187us) server_negotiation.cc:407] Connection header received
0924 07:16:24.809551 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.809552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.809560 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.809608 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.810175 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.810674 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.811591 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.811791 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.811852 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.813974 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.813976 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.813976 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.813983 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.815166 (+ 1183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.815510 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.815511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.815511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.815561 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.815763 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.815764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.815764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.815868 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.815921 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.816194 (+ 273us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.816272 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:16:24.816502 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.808892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:44777 (local address 127.0.0.1:52566)
0924 07:16:24.809031 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.809070 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:24.809483 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:24.809659 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:24.809662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:24.809893 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:24.810083 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.810088 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.810760 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.810761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.811515 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.811517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.811854 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.811855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.811963 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.812447 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:24.812455 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:24.813894 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:24.815233 (+ 1339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.815239 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.815241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.815447 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.815611 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.815612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.815612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.815712 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.815920 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:24.815926 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:24.816085 (+ 159us) client_negotiation.cc:769] Sending connection context
0924 07:16:24.816169 (+ 84us) client_negotiation.cc:240] Negotiation successful
0924 07:16:24.816306 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for kudu/127.10.130.252@KRBTEST.COM; defaulting to no policy
I20260924 07:16:24.816978 24172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.252@KRBTEST.COM" created.
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.253@KRBTEST.COM for kudu/127.10.130.253@KRBTEST.COM
I20260924 07:16:24.823414 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.817189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:41597 (local address 127.0.0.1:60966)
0924 07:16:24.817229 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.817237 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:24.817269 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:24.817531 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:24.817532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:24.817653 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:24.817696 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.817697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.818550 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.818551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.819111 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.819113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.819518 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.819519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.819575 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.819931 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:24.819936 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:24.821073 (+ 1137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:24.822536 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.822537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.822538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.822681 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.822842 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.822842 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.822842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.822893 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.823130 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:24.823131 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:24.823165 (+ 34us) client_negotiation.cc:769] Sending connection context
0924 07:16:24.823223 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 07:16:24.823308 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260924 07:16:24.823472 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.817262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60966 (local address 127.10.130.253:41597)
0924 07:16:24.817394 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.817397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.817412 (+ 15us) server_negotiation.cc:407] Connection header received
0924 07:16:24.817427 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.817428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.817437 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.817529 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.817754 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.818477 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.819173 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.819462 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.819516 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.821142 (+ 1626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.821148 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.821149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.821155 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.822479 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.822742 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.822745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.822745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.822793 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.822941 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.822942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.822942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.823077 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.823142 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.823248 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.823356 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260924 07:16:24.824290 24172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:16:24.824553 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.824466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:40675 (local address 127.0.0.1:41506)
0924 07:16:24.824494 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.824533 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260924 07:16:24.824662 24120 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.130.252:40675: connect: Connection refused (error 111)
I20260924 07:16:24.825007 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.824928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:40675 (local address 127.0.0.1:41512)
0924 07:16:24.824965 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.824987 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260924 07:16:24.825093 24172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.252:40675: Network error: Client connection negotiation failed: client connection to 127.10.130.252:40675: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.252.keytab.
Entry for principal kudu/127.10.130.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.252.keytab.
I20260924 07:16:24.831717 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.252:40675
--webserver_interface=127.10.130.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.252.keytab
--principal=kudu/127.10.130.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:44777,127.10.130.253:41597,127.10.130.252:40675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:16:24.874291 24172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } attempt: 1
I20260924 07:16:24.874787 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.874659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:40675 (local address 127.0.0.1:41526)
0924 07:16:24.874715 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.874756 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 07:16:24.875119 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.875037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:40675 (local address 127.0.0.1:41538)
0924 07:16:24.875078 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.875100 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 07:16:24.875200 24172 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.252:40675: Network error: Client connection negotiation failed: client connection to 127.10.130.252:40675: connect: Connection refused (error 111)
I20260924 07:16:24.895675 24097 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } attempt: 3
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for kudu/127.10.130.253@KRBTEST.COM
I20260924 07:16:24.902144 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.896090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60982 (local address 127.10.130.253:41597)
0924 07:16:24.896152 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.896154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.896164 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:16:24.896191 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.896192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.896204 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.896250 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.896560 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.897053 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.897699 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.897915 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.897968 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.899910 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.899912 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.899912 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.899919 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.901183 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.901475 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.901476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.901477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.901525 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.901681 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.901682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.901682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.901787 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.901848 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.901973 (+ 125us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.902044 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 07:16:24.902144 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.896025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:41597 (local address 127.0.0.1:60982)
0924 07:16:24.896079 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.896090 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:24.896137 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:24.896260 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:24.896261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:24.896421 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:24.896483 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.896485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.897122 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.897123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.897626 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.897628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.897972 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.897973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.898040 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.898515 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:24.898519 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:24.899830 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:24.901255 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.901256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.901256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.901417 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.901576 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.901576 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.901577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.901633 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.901836 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:24.901837 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:24.901885 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 07:16:24.901946 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 07:16:24.902044 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 07:16:24.902607 24097 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:16:24.902931 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.902839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:40675 (local address 127.0.0.1:41550)
0924 07:16:24.902877 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.902904 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260924 07:16:24.903189 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.903115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:40675 (local address 127.0.0.1:41558)
0924 07:16:24.903147 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.903168 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260924 07:16:24.903265 24097 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.252:40675: Network error: Client connection negotiation failed: client connection to 127.10.130.252:40675: connect: Connection refused (error 111)
W20260924 07:16:24.910266 24179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:24.910454 24179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:24.910482 24179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:24.911612 24179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:16:24.911648 24179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:24.911666 24179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:24.911684 24179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:16:24.911706 24179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:16:24.913168 24179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:44777,127.10.130.253:41597,127.10.130.252:40675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.252.keytab
--principal=kudu/127.10.130.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.252:40675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.130.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:24.913398 24179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:24.913594 24179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:24.917997 24187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:24.918005 24184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:24.918006 24185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:24.918049 24179 server_base.cc:1034] running on GCE node
I20260924 07:16:24.918412 24179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:24.918671 24179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:24.919803 24179 hybrid_clock.cc:648] HybridClock initialized: now 1790234184919791 us; error 28 us; skew 500 ppm
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:24.922108 24179 init.cc:377] Logged in from keytab as kudu/127.10.130.252@KRBTEST.COM (short username kudu)
I20260924 07:16:24.922547 24179 webserver.cc:460] Webserver started at http://127.10.130.252:32783/ using document root <none> and password file <none>
I20260924 07:16:24.922752 24179 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:24.922799 24179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:24.922919 24179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:16:24.923926 24179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/data/instance:
uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d"
format_stamp: "Formatted at 2026-09-24 07:16:24 on dist-test-slave-7q7m"
server_key: "fa956c103eadd3175603cfa6b9724443"
server_key_iv: "d7176d9bcd654279408984200e40cfcc"
server_key_version: "encryptionkey@0"
I20260924 07:16:24.924291 24179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/wal/instance:
uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d"
format_stamp: "Formatted at 2026-09-24 07:16:24 on dist-test-slave-7q7m"
server_key: "fa956c103eadd3175603cfa6b9724443"
server_key_iv: "d7176d9bcd654279408984200e40cfcc"
server_key_version: "encryptionkey@0"
I20260924 07:16:24.925478 24179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:16:24.926101 24194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:24.926240 24179 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:24.926321 24179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/wal
uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d"
format_stamp: "Formatted at 2026-09-24 07:16:24 on dist-test-slave-7q7m"
server_key: "fa956c103eadd3175603cfa6b9724443"
server_key_iv: "d7176d9bcd654279408984200e40cfcc"
server_key_version: "encryptionkey@0"
I20260924 07:16:24.926385 24179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:24.933398 24179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:24.934391 24097 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } attempt: 1
I20260924 07:16:24.934738 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.934627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:40675 (local address 127.0.0.1:41560)
0924 07:16:24.934677 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.934711 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 07:16:24.935058 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.934968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:40675 (local address 127.0.0.1:41574)
0924 07:16:24.935003 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.935029 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260924 07:16:24.935179 24097 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.252:40675: Network error: Client connection negotiation failed: client connection to 127.10.130.252:40675: connect: Connection refused (error 111)
I20260924 07:16:24.935583 24179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:24.935709 24179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:24.939289 24179 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.252:40675
I20260924 07:16:24.939325 24246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.252:40675 every 8 connection(s)
I20260924 07:16:24.939644 24179 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/data/info.pb
I20260924 07:16:24.940425 24247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:24.941907 24247 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:16:24.943315 24172 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } attempt: 2
I20260924 07:16:24.945443 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 24179
I20260924 07:16:24.945533 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790234044967003-10763-0/minicluster-data/master-2/wal/instance
I20260924 07:16:24.945703 10763 external_mini_cluster.cc:1428] Setting key d0bf463a1487f93d7c29e58c93586e69
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.253@KRBTEST.COM for kudu/127.10.130.252@KRBTEST.COM
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.252@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.252@KRBTEST.COM
I20260924 07:16:24.951344 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.942534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52568 (local address 127.10.130.254:44777)
0924 07:16:24.942598 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.942600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.942750 (+ 150us) server_negotiation.cc:407] Connection header received
0924 07:16:24.943200 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.943201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.943210 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.943257 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.943893 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.944552 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.945777 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.946079 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.946140 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.948950 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.948951 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.948951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.948958 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.950235 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.950571 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.950573 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.950573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.950629 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.950818 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.950818 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.950819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.950926 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.950986 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.951176 (+ 190us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.951251 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 07:16:24.951350 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.943547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:40675 (local address 127.0.0.1:41588)
0924 07:16:24.943604 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.943615 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:24.943660 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:24.943861 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:24.943862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:24.944042 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:24.944087 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.944090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.945160 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.945161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.945885 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.945888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.946474 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.946475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.946539 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.946949 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:24.946954 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:24.948149 (+ 1195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:24.950126 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.950127 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.950127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.950326 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.950591 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.950591 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.950592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.950653 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.951057 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:24.951058 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:24.951101 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 07:16:24.951160 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 07:16:24.951258 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 07:16:24.951555 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.942512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:44777 (local address 127.0.0.1:52568)
0924 07:16:24.942643 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.942677 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:24.943127 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:24.943309 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:24.943312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:24.943579 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:24.943812 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.943817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.944632 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.944632 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.945680 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.945682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.946196 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.946197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.946310 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.946903 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:24.946913 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:24.948869 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:24.950311 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.950318 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.950320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.950507 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.950681 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.950682 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.950682 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.950766 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.950985 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:24.950991 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:24.951058 (+ 67us) client_negotiation.cc:769] Sending connection context
0924 07:16:24.951149 (+ 91us) client_negotiation.cc:240] Negotiation successful
0924 07:16:24.951281 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":117,"thread_start_us":69,"threads_started":1}
I20260924 07:16:24.951691 24250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.943597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41588 (local address 127.10.130.252:40675)
0924 07:16:24.943726 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.943731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.943747 (+ 16us) server_negotiation.cc:407] Connection header received
0924 07:16:24.943767 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.943769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.943786 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.943845 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.944158 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.945092 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.945966 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.946397 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.946464 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.948218 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.948226 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.948226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.948232 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.950027 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.950416 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.950419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.950419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.950514 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.950718 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.950719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.950719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.950989 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.951066 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.951532 (+ 466us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.951599 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260924 07:16:24.953474 24172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687: Bootstrap starting.
I20260924 07:16:24.954440 24172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:24.954454 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.946167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41590 (local address 127.10.130.252:40675)
0924 07:16:24.946296 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.946298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.946304 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:16:24.946320 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.946321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.946330 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.946383 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.947052 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.947665 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.948698 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.948987 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.949045 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.951547 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.951548 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.951549 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.951555 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.953279 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.953647 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.953648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.953649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.953702 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.953890 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.953890 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.953891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.954023 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.954150 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.954266 (+ 116us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.954341 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260924 07:16:24.954733 24172 log.cc:824] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:24.955209 24247 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:16:24.955549 24172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687: No bootstrap required, opened a new log
I20260924 07:16:24.957540 24172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } }
I20260924 07:16:24.957667 24172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:24.957703 24172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e1e258075b64a918ac2e12b11456687, State: Initialized, Role: FOLLOWER
I20260924 07:16:24.957823 24172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } }
I20260924 07:16:24.958128 24254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } } }
I20260924 07:16:24.958235 24254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:16:24.958379 24172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [sys.catalog]: configured and running, proceeding with master startup.
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.252@KRBTEST.COM for kudu/127.10.130.253@KRBTEST.COM
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
W20260924 07:16:24.960527 24265 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:16:24.960631 24265 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:16:24.961588 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.955422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:41597 (local address 127.0.0.1:60990)
0924 07:16:24.955464 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.955472 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:24.955503 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:24.955600 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:24.955601 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:24.955715 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:24.955759 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.955760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.956385 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.956386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.956853 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.956854 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.957163 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.957163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.957227 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.957557 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:24.957562 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:24.958746 (+ 1184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:24.960375 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.960375 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.960376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.960575 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.960766 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.960767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.960767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.960818 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.961320 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:24.961321 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:24.961364 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 07:16:24.961421 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 07:16:24.961503 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 07:16:24.962229 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.955473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60990 (local address 127.10.130.253:41597)
0924 07:16:24.955507 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.955509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.955533 (+ 24us) server_negotiation.cc:407] Connection header received
0924 07:16:24.955550 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.955550 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.955558 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.955597 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.955819 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.956312 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.956913 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.957111 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.957162 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.958815 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.958816 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.958817 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.958823 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.960289 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.960636 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.960637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.960638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.960691 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.960876 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.960877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.960877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.961233 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.961293 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.962083 (+ 790us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.962154 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:16:24.962508 24247 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:16:24.960542 24104 master_runner.cc:186] Error getting master registration for 127.10.130.254:44777: OK, instance_id { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" instance_seqno: 1790234184670410 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 07:16:24.962781 24104 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 07:16:24.962965 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.954990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52574 (local address 127.10.130.254:44777)
0924 07:16:24.955031 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.955032 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.955041 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:16:24.955062 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.955063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.955073 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.955137 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.955645 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.956324 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.957248 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.957525 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.957568 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.960068 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.960069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.960069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.960075 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.961733 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.962097 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.962098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.962099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.962155 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.962339 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.962339 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.962340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.962463 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.962599 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.962810 (+ 211us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.962884 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.252@KRBTEST.COM for kudu/127.10.130.252@KRBTEST.COM
I20260924 07:16:24.968830 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.962955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:40675 (local address 127.0.0.1:41604)
0924 07:16:24.963044 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:24.963054 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:24.963098 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:24.963231 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:24.963232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:24.963363 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:24.963396 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.963398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.964211 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.964212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.964760 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:24.964762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.965123 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.965124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:24.965157 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.965493 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:24.965497 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:24.966491 (+ 994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:24.967871 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.967872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.967872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.968105 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.968260 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:24.968261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:24.968261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:24.968312 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:24.968545 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:24.968546 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:24.968583 (+ 37us) client_negotiation.cc:769] Sending connection context
0924 07:16:24.968643 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 07:16:24.968730 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260924 07:16:24.968878 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.963005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41604 (local address 127.10.130.252:40675)
0924 07:16:24.963044 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.963046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.963142 (+ 96us) server_negotiation.cc:407] Connection header received
0924 07:16:24.963157 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.963157 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.963166 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.963207 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.963474 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.964151 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.964823 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.965031 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.965085 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.966559 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.966560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.966560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.966566 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.967792 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.968163 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.968164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.968164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.968211 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.968358 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.968359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.968359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.968490 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.968546 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.968667 (+ 121us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.968779 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":19}
Sep 24 07:16:24 dist-test-slave-7q7m krb5kdc[24016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.253@KRBTEST.COM
I20260924 07:16:24.970443 24247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d: Bootstrap starting.
I20260924 07:16:24.971040 24247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:24.971306 24247 log.cc:824] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:24.972121 24247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d: No bootstrap required, opened a new log
I20260924 07:16:24.972168 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.964281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60994 (local address 127.10.130.253:41597)
0924 07:16:24.964316 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.964318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.964325 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:16:24.964367 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.964368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.964376 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.964424 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.964971 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.965776 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.966709 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.966996 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.967033 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.969529 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.969530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.969531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.969536 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.971017 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.971384 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.971385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.971386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.971447 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.971639 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.971640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.971641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.971775 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.971963 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.972023 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.972088 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:16:24.974081 24247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } }
I20260924 07:16:24.974236 24247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:24.974274 24247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fcccf81f1ca4fc6bba8e5505c726c8d, State: Initialized, Role: FOLLOWER
I20260924 07:16:24.974392 24247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } }
I20260924 07:16:24.974630 24268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } } }
I20260924 07:16:24.974689 24247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:16:24.974713 24268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [sys.catalog]: This master's current role is: FOLLOWER
W20260924 07:16:24.976619 24279 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:16:24.976715 24279 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:16:24.976780 24179 master_runner.cc:186] Error getting master registration for 127.10.130.254:44777: OK, instance_id { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" instance_seqno: 1790234184670410 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 07:16:24.978276 24179 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 07:16:24.979282 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:16:24.988313 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.980548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32768 (local address 127.10.130.253:41597)
0924 07:16:24.980587 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.980589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.980982 (+ 393us) server_negotiation.cc:407] Connection header received
0924 07:16:24.981584 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.981585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.981594 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.981620 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.982260 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.982947 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.983839 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.984133 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.984172 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.985458 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.985460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.985460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.985466 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.986997 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.987543 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.987544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.987545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.987600 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.987778 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.987778 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.987779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.987887 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.987949 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.988154 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.988227 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:16:24.989563 24100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.980854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52580 (local address 127.10.130.254:44777)
0924 07:16:24.980887 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.980889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.980896 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:16:24.980917 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.980918 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.980927 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.980962 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.981522 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.982230 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.983202 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.983501 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.983540 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.985283 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.985284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.985284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.985291 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.987098 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.988762 (+ 1664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.988764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.988764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.988826 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.989027 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.989028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.989028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.989149 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.989212 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.989398 (+ 186us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.989479 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:16:24.993253 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:24.980842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41610 (local address 127.10.130.252:40675)
0924 07:16:24.980878 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:24.980880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:24.984538 (+ 3658us) server_negotiation.cc:407] Connection header received
0924 07:16:24.984642 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:24.984643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:24.984654 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:24.984682 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:24.987389 (+ 2707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.988096 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.989551 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:24.989816 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:24.989855 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:24.991070 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:24.991071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:24.991071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:24.991077 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:24.992289 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.992601 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.992602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.992602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.992656 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:24.992818 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:24.992819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:24.992819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:24.992918 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:24.993016 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:24.993109 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 07:16:24.993171 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:16:25.010339 24097 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } attempt: 2
Sep 24 07:16:25 dist-test-slave-7q7m krb5kdc[24016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234184, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for kudu/127.10.130.252@KRBTEST.COM
I20260924 07:16:25.016664 24099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:25.010605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:40675 (local address 127.0.0.1:41626)
0924 07:16:25.010654 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:25.010665 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:25.010712 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:25.010823 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:25.010824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:25.010970 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:25.011008 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:25.011009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:25.011579 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:25.011580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:25.012087 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:25.012089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:25.012384 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:25.012385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:25.012444 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:25.012838 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:25.012843 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:25.014349 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:25.015730 (+ 1381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:25.015731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:25.015731 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:25.015888 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:25.016100 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:25.016101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:25.016101 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:25.016158 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:25.016361 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:25.016362 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:25.016407 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 07:16:25.016472 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 07:16:25.016568 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 07:16:25.016664 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:25.010640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.10.130.252:40675)
0924 07:16:25.010673 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:25.010674 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:25.010698 (+ 24us) server_negotiation.cc:407] Connection header received
0924 07:16:25.010763 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:25.010764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:25.010774 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:25.010816 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:25.011069 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:25.011514 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:25.012160 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:25.012333 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:25.012383 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:25.014420 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:25.014422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:25.014422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:25.014429 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:25.015652 (+ 1223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:25.015951 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:25.015952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:25.015952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:25.016046 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:25.016207 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:25.016208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:25.016208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:25.016310 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:25.016362 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:25.016500 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 07:16:25.016571 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:16:25.018206 24097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1: Bootstrap starting.
I20260924 07:16:25.018810 24097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:25.019073 24097 log.cc:824] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:25.019770 24097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1: No bootstrap required, opened a new log
I20260924 07:16:25.021153 24097 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } }
I20260924 07:16:25.021260 24097 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:25.021293 24097 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f07390429354e1da1aa6f70a2387ba1, State: Initialized, Role: FOLLOWER
I20260924 07:16:25.021466 24097 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } }
I20260924 07:16:25.021697 24289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } } }
I20260924 07:16:25.021757 24097 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:16:25.021822 24289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 07:16:25.023828 24300 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:16:25.023876 24300 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:16:25.025682 24029 master_runner.cc:263] No leader master found from master 1f07390429354e1da1aa6f70a2387ba1
I20260924 07:16:25.025724 24029 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 07:16:25.052330 24125 tablet.cc:2369] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:16:25.147135 24268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:16:25.147255 24268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } }
I20260924 07:16:25.147491 24268 leader_election.cc:288] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f07390429354e1da1aa6f70a2387ba1 (127.10.130.254:44777), 3e1e258075b64a918ac2e12b11456687 (127.10.130.253:41597)
I20260924 07:16:25.147763 24071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f07390429354e1da1aa6f70a2387ba1" is_pre_election: true
I20260924 07:16:25.147754 24146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e1e258075b64a918ac2e12b11456687" is_pre_election: true
I20260924 07:16:25.147895 24146 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fcccf81f1ca4fc6bba8e5505c726c8d in term 0.
I20260924 07:16:25.147895 24071 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fcccf81f1ca4fc6bba8e5505c726c8d in term 0.
I20260924 07:16:25.148100 24196 leader_election.cc:302] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e1e258075b64a918ac2e12b11456687, 4fcccf81f1ca4fc6bba8e5505c726c8d; no voters:
I20260924 07:16:25.148232 24268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:16:25.148280 24268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:16:25.148299 24268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:25.148947 24268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } }
I20260924 07:16:25.149053 24268 leader_election.cc:288] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [CANDIDATE]: Term 1 election: Requested vote from peers 1f07390429354e1da1aa6f70a2387ba1 (127.10.130.254:44777), 3e1e258075b64a918ac2e12b11456687 (127.10.130.253:41597)
I20260924 07:16:25.149211 24146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e1e258075b64a918ac2e12b11456687"
I20260924 07:16:25.149209 24071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f07390429354e1da1aa6f70a2387ba1"
I20260924 07:16:25.149284 24146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:25.149299 24071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:25.149906 24146 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fcccf81f1ca4fc6bba8e5505c726c8d in term 1.
I20260924 07:16:25.150036 24196 leader_election.cc:302] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e1e258075b64a918ac2e12b11456687, 4fcccf81f1ca4fc6bba8e5505c726c8d; no voters:
I20260924 07:16:25.150123 24268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:16:25.150146 24071 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fcccf81f1ca4fc6bba8e5505c726c8d in term 1.
I20260924 07:16:25.150274 24268 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [term 1 LEADER]: Becoming Leader. State: Replica: 4fcccf81f1ca4fc6bba8e5505c726c8d, State: Running, Role: LEADER
I20260924 07:16:25.150362 24268 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } }
I20260924 07:16:25.150779 24302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fcccf81f1ca4fc6bba8e5505c726c8d. Latest consensus state: current_term: 1 leader_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } } }
I20260924 07:16:25.150869 24302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:25.151050 24304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:16:25.151221 24304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:16:25.152508 24146 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [term 1 FOLLOWER]: Refusing update from remote peer 4fcccf81f1ca4fc6bba8e5505c726c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:16:25.152733 24071 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [term 1 FOLLOWER]: Refusing update from remote peer 4fcccf81f1ca4fc6bba8e5505c726c8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:16:25.152765 24302 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:16:25.152947 24268 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:16:25.154039 24254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fcccf81f1ca4fc6bba8e5505c726c8d. Latest consensus state: current_term: 1 leader_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } } }
I20260924 07:16:25.154042 24289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fcccf81f1ca4fc6bba8e5505c726c8d. Latest consensus state: current_term: 1 leader_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } } }
I20260924 07:16:25.154115 24289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:16:25.154114 24254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:16:25.154903 24268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } } }
I20260924 07:16:25.154989 24268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:25.155288 24268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } } }
I20260924 07:16:25.155360 24268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:25.155390 24289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } } }
I20260924 07:16:25.155462 24289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:16:25.155553 24304 catalog_manager.cc:1347] Generated new cluster ID: 42742ff97abf4c01a11bcca33cab4630
I20260924 07:16:25.155596 24304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:16:25.155648 24307 mvcc.cc:205] Tried to move back new op lower bound from 7332799222382628864 to 7332799222376505344. Current Snapshot: MvccSnapshot[applied={T|T < 7332799222382628864}]
I20260924 07:16:25.155768 24254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f07390429354e1da1aa6f70a2387ba1" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44777 } } peers { permanent_uuid: "3e1e258075b64a918ac2e12b11456687" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 41597 } } peers { permanent_uuid: "4fcccf81f1ca4fc6bba8e5505c726c8d" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 40675 } } }
I20260924 07:16:25.155841 24254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:16:25.160104 24305 mvcc.cc:205] Tried to move back new op lower bound from 7332799222382628864 to 7332799222376505344. Current Snapshot: MvccSnapshot[applied={T|T < 7332799222382628864 or (T in {7332799222382628864})}]
I20260924 07:16:25.160197 24306 mvcc.cc:205] Tried to move back new op lower bound from 7332799222382628864 to 7332799222376505344. Current Snapshot: MvccSnapshot[applied={T|T < 7332799222382628864}]
I20260924 07:16:25.172832 24304 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:16:25.173370 24304 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:16:25.177263 24304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fcccf81f1ca4fc6bba8e5505c726c8d: Generated new TSK 0
I20260924 07:16:25.177449 24304 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:16:25.961079 24265 catalog_manager.cc:1259] Loaded cluster ID: 42742ff97abf4c01a11bcca33cab4630
I20260924 07:16:25.961149 24265 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687: loading cluster ID for follower catalog manager: success
I20260924 07:16:25.961910 24265 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687: acquiring CA information for follower catalog manager: success
I20260924 07:16:25.962153 24265 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3e1e258075b64a918ac2e12b11456687: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 07:16:26.024274 24300 catalog_manager.cc:1259] Loaded cluster ID: 42742ff97abf4c01a11bcca33cab4630
I20260924 07:16:26.024353 24300 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1: loading cluster ID for follower catalog manager: success
I20260924 07:16:26.025187 24300 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1: acquiring CA information for follower catalog manager: success
I20260924 07:16:26.025490 24300 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1f07390429354e1da1aa6f70a2387ba1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 07:16:26.225692 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 24029
I20260924 07:16:26.230254 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 24104
I20260924 07:16:26.234440 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 24179
2026-09-24T07:16:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2730 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:16:26 dist-test-slave-7q7m krb5kdc[24320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:16:26 dist-test-slave-7q7m krb5kdc[24320](info): set up 2 sockets
Sep 24 07:16:26 dist-test-slave-7q7m krb5kdc[24320](info): commencing operation
krb5kdc: starting...
W20260924 07:16:27.179674 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:16:27 dist-test-slave-7q7m krb5kdc[24320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:16:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:16:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:16:27.284487 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:42857
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:42857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:27.361483 24333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:27.361696 24333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:27.361726 24333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:27.362885 24333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:16:27.362931 24333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:27.362948 24333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:27.362963 24333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:16:27.362978 24333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:16:27.364480 24333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:42857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:42857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:27.364728 24333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:27.364939 24333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:27.369470 24339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:27.369520 24333 server_base.cc:1034] running on GCE node
W20260924 07:16:27.369465 24341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:27.369623 24338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:27.369855 24333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:27.370162 24333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:27.371328 24333 hybrid_clock.cc:648] HybridClock initialized: now 1790234187371315 us; error 28 us; skew 500 ppm
Sep 24 07:16:27 dist-test-slave-7q7m krb5kdc[24320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234187, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:27.373628 24333 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:16:27.374043 24333 webserver.cc:460] Webserver started at http://127.10.130.254:40627/ using document root <none> and password file <none>
I20260924 07:16:27.374233 24333 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:27.374272 24333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:27.374368 24333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:16:27.375370 24333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "1aae8bd7c38a4d45b27ce7447578c047"
format_stamp: "Formatted at 2026-09-24 07:16:27 on dist-test-slave-7q7m"
server_key: "f3d622597f6d04f565ae85421a08b6ef"
server_key_iv: "987bd0682f2f70af347d098aeb099aa1"
server_key_version: "encryptionkey@0"
I20260924 07:16:27.375661 24333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "1aae8bd7c38a4d45b27ce7447578c047"
format_stamp: "Formatted at 2026-09-24 07:16:27 on dist-test-slave-7q7m"
server_key: "f3d622597f6d04f565ae85421a08b6ef"
server_key_iv: "987bd0682f2f70af347d098aeb099aa1"
server_key_version: "encryptionkey@0"
I20260924 07:16:27.376899 24333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:16:27.377545 24348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:27.377719 24333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:16:27.377794 24333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "1aae8bd7c38a4d45b27ce7447578c047"
format_stamp: "Formatted at 2026-09-24 07:16:27 on dist-test-slave-7q7m"
server_key: "f3d622597f6d04f565ae85421a08b6ef"
server_key_iv: "987bd0682f2f70af347d098aeb099aa1"
server_key_version: "encryptionkey@0"
I20260924 07:16:27.377871 24333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:27.385172 24333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:27.386744 24333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:27.386847 24333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:27.390479 24333 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:42857
I20260924 07:16:27.390512 24400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:42857 every 8 connection(s)
I20260924 07:16:27.390830 24333 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:16:27.391628 24401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:27.393841 24401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047: Bootstrap starting.
I20260924 07:16:27.394366 24401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:27.394610 24401 log.cc:824] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:27.395215 24401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047: No bootstrap required, opened a new log
I20260924 07:16:27.396807 24401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aae8bd7c38a4d45b27ce7447578c047" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42857 } }
I20260924 07:16:27.396938 24401 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:27.396960 24401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aae8bd7c38a4d45b27ce7447578c047, State: Initialized, Role: FOLLOWER
I20260924 07:16:27.397100 24401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aae8bd7c38a4d45b27ce7447578c047" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42857 } }
I20260924 07:16:27.397161 24401 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:16:27.397181 24401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:16:27.397228 24401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:27.397696 24401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aae8bd7c38a4d45b27ce7447578c047" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42857 } }
I20260924 07:16:27.397783 24401 leader_election.cc:302] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aae8bd7c38a4d45b27ce7447578c047; no voters:
I20260924 07:16:27.397951 24401 leader_election.cc:288] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:16:27.397979 24404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:16:27.398128 24404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [term 1 LEADER]: Becoming Leader. State: Replica: 1aae8bd7c38a4d45b27ce7447578c047, State: Running, Role: LEADER
I20260924 07:16:27.398175 24401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:16:27.398209 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 24333
I20260924 07:16:27.398252 24404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aae8bd7c38a4d45b27ce7447578c047" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42857 } }
I20260924 07:16:27.398303 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:16:27.398484 10763 external_mini_cluster.cc:1428] Setting key d9fc087355472edf4f84af6830229cc5
I20260924 07:16:27.398983 24406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1aae8bd7c38a4d45b27ce7447578c047" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aae8bd7c38a4d45b27ce7447578c047" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42857 } } }
I20260924 07:16:27.399070 24406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:27.399088 24405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aae8bd7c38a4d45b27ce7447578c047. Latest consensus state: current_term: 1 leader_uuid: "1aae8bd7c38a4d45b27ce7447578c047" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aae8bd7c38a4d45b27ce7447578c047" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42857 } } }
I20260924 07:16:27.399133 24405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047 [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:27.399798 24415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:16:27.400295 24415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:16:27.401525 24415 catalog_manager.cc:1347] Generated new cluster ID: 04060553be16423088cc0e8f06c30ce2
I20260924 07:16:27.401568 24415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:16:27 dist-test-slave-7q7m krb5kdc[24320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:27.409873 24414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:27.399226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.10.130.254:42857)
0924 07:16:27.399408 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:27.399418 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:27.399452 (+ 34us) server_negotiation.cc:407] Connection header received
0924 07:16:27.399564 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:27.399572 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:27.399632 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:27.400572 (+ 940us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:27.400664 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:27.401995 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:27.403025 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:27.403644 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:27.403755 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:27.406386 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:27.406390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:27.406391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:27.406403 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:27.408296 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:27.408679 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:27.408681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:27.408682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:27.408800 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:27.408982 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:27.408982 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:27.408982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:27.409264 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:27.409398 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:27.409512 (+ 114us) server_negotiation.cc:299] Negotiation successful
0924 07:16:27.409641 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260924 07:16:27.410729 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:16:27.412598 24415 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:16:27.413297 24415 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:16:27.432122 24415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1aae8bd7c38a4d45b27ce7447578c047: Generated new TSK 0
I20260924 07:16:27.432374 24415 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:16:27.464697 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 24333
2026-09-24T07:16:27Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1229 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:16:27 dist-test-slave-7q7m krb5kdc[24428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:16:27 dist-test-slave-7q7m krb5kdc[24428](info): set up 2 sockets
Sep 24 07:16:27 dist-test-slave-7q7m krb5kdc[24428](info): commencing operation
krb5kdc: starting...
W20260924 07:16:28.399767 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.918s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:16:28 dist-test-slave-7q7m krb5kdc[24428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:16:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:16:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:16:28.508549 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39257
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:39257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:28.585963 24441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:28.586169 24441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:28.586197 24441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:28.587317 24441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:16:28.587357 24441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:28.587378 24441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:28.587392 24441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:16:28.587407 24441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:16:28.588882 24441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:39257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:28.589120 24441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:28.589318 24441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:28.593743 24449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:28.593775 24447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:28.593904 24441 server_base.cc:1034] running on GCE node
W20260924 07:16:28.593772 24446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:28.594168 24441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:28.594437 24441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:28.595571 24441 hybrid_clock.cc:648] HybridClock initialized: now 1790234188595561 us; error 27 us; skew 500 ppm
Sep 24 07:16:28 dist-test-slave-7q7m krb5kdc[24428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234188, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:28.597788 24441 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:16:28.598166 24441 webserver.cc:460] Webserver started at http://127.10.130.254:39407/ using document root <none> and password file <none>
I20260924 07:16:28.598349 24441 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:28.598395 24441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:28.598513 24441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:16:28.599509 24441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "59a88f46ff3c47b285844a5c4e454849"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "a5a5892fa36e19c5d8bf86c8688704d3"
server_key_iv: "67ff218427a8ace435436f529b98a9da"
server_key_version: "encryptionkey@0"
I20260924 07:16:28.599833 24441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "59a88f46ff3c47b285844a5c4e454849"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "a5a5892fa36e19c5d8bf86c8688704d3"
server_key_iv: "67ff218427a8ace435436f529b98a9da"
server_key_version: "encryptionkey@0"
I20260924 07:16:28.601055 24441 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 07:16:28.601681 24456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:28.601821 24441 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:16:28.601899 24441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "59a88f46ff3c47b285844a5c4e454849"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "a5a5892fa36e19c5d8bf86c8688704d3"
server_key_iv: "67ff218427a8ace435436f529b98a9da"
server_key_version: "encryptionkey@0"
I20260924 07:16:28.601969 24441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:28.623358 24441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:28.625073 24441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:28.625178 24441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:28.628744 24441 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:39257
I20260924 07:16:28.628803 24508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:39257 every 8 connection(s)
I20260924 07:16:28.629101 24441 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:16:28.629910 24509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:28.632148 24509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Bootstrap starting.
I20260924 07:16:28.632449 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 24441
I20260924 07:16:28.632529 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:16:28.632712 10763 external_mini_cluster.cc:1428] Setting key 8f8fa305894433eff295ace242ad2ef9
I20260924 07:16:28.632725 24509 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:28.632967 24509 log.cc:824] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:28.633848 24509 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: No bootstrap required, opened a new log
I20260924 07:16:28.636011 24509 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } }
I20260924 07:16:28.636191 24509 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:28.636232 24509 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59a88f46ff3c47b285844a5c4e454849, State: Initialized, Role: FOLLOWER
I20260924 07:16:28.636356 24509 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } }
I20260924 07:16:28.636442 24509 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:16:28.636473 24509 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:16:28.636520 24509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:28.637099 24509 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } }
I20260924 07:16:28.637203 24509 leader_election.cc:302] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59a88f46ff3c47b285844a5c4e454849; no voters:
I20260924 07:16:28.637355 24509 leader_election.cc:288] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:16:28.637395 24514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:16:28.637530 24514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 1 LEADER]: Becoming Leader. State: Replica: 59a88f46ff3c47b285844a5c4e454849, State: Running, Role: LEADER
I20260924 07:16:28.637586 24509 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:16:28.637657 24514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } }
I20260924 07:16:28.637977 24514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59a88f46ff3c47b285844a5c4e454849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } } }
I20260924 07:16:28.638101 24514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:28.638281 24514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59a88f46ff3c47b285844a5c4e454849. Latest consensus state: current_term: 1 leader_uuid: "59a88f46ff3c47b285844a5c4e454849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } } }
I20260924 07:16:28.638386 24514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:28.638716 24518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:16:28.639189 24518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 07:16:28 dist-test-slave-7q7m krb5kdc[24428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:28.640748 24518 catalog_manager.cc:1347] Generated new cluster ID: 93c58b2222a84c519ab04d7b100b196d
I20260924 07:16:28.640797 24518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:16:28.643611 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:28.633149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36206 (local address 127.10.130.254:39257)
0924 07:16:28.633321 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:28.633330 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:28.633363 (+ 33us) server_negotiation.cc:407] Connection header received
0924 07:16:28.633884 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:28.633894 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:28.633957 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:28.634108 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:28.634658 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:28.636038 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:28.636839 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:28.637445 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:28.637553 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:28.640182 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:28.640185 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:28.640185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:28.640197 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:28.642088 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:28.642450 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:28.642453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:28.642454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:28.642561 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:28.642731 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:28.642731 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:28.642732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:28.643039 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:28.643198 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:28.643278 (+ 80us) server_negotiation.cc:299] Negotiation successful
0924 07:16:28.643408 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":63,"threads_started":1}
I20260924 07:16:28.652372 24518 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:16:28.653052 24518 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:16:28.656919 24518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Generated new TSK 0
I20260924 07:16:28.657087 24518 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:16:28.679840 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39257
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:28.758054 24535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:28.758253 24535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:28.758275 24535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:28.759395 24535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:28.759434 24535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:28.759459 24535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:16:28.760911 24535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:28.761135 24535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:28.761341 24535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 07:16:28.765954 24535 server_base.cc:1034] running on GCE node
W20260924 07:16:28.765892 24543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:28.765919 24540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:28.765892 24541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:28.766335 24535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:28.766553 24535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:28.767694 24535 hybrid_clock.cc:648] HybridClock initialized: now 1790234188767674 us; error 31 us; skew 500 ppm
Sep 24 07:16:28 dist-test-slave-7q7m krb5kdc[24428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234188, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:28.769963 24535 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:16:28.770346 24535 webserver.cc:460] Webserver started at http://127.10.130.193:41365/ using document root <none> and password file <none>
I20260924 07:16:28.770541 24535 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:28.770594 24535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:28.770714 24535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:16:28.771693 24535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "418086cdd66044d7b0fa1027e82dbf77"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "50a962637fe1c0b136ec64f4cf5922e4"
server_key_iv: "d26f93a42ce3ed480bd4cbfec6baf25f"
server_key_version: "encryptionkey@0"
I20260924 07:16:28.772078 24535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "418086cdd66044d7b0fa1027e82dbf77"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "50a962637fe1c0b136ec64f4cf5922e4"
server_key_iv: "d26f93a42ce3ed480bd4cbfec6baf25f"
server_key_version: "encryptionkey@0"
I20260924 07:16:28.773247 24535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:16:28.773900 24550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:28.774041 24535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:16:28.774128 24535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "418086cdd66044d7b0fa1027e82dbf77"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "50a962637fe1c0b136ec64f4cf5922e4"
server_key_iv: "d26f93a42ce3ed480bd4cbfec6baf25f"
server_key_version: "encryptionkey@0"
I20260924 07:16:28.774189 24535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:28.797008 24535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:28.798769 24535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:28.798872 24535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:28.799038 24535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:28.799297 24535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:28.799325 24535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:28.799346 24535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:28.799362 24535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:28.804725 24535 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:40785
I20260924 07:16:28.804790 24663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:40785 every 8 connection(s)
I20260924 07:16:28.805090 24535 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
Sep 24 07:16:28 dist-test-slave-7q7m krb5kdc[24428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234188, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:28.813665 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:28.805834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:42919 (local address 127.10.130.254:39257)
0924 07:16:28.805915 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:28.805917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:28.806110 (+ 193us) server_negotiation.cc:407] Connection header received
0924 07:16:28.806627 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:28.806628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:28.806641 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:28.806691 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:28.807647 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:28.808213 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:28.809153 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:28.809359 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:28.809413 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:28.811368 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:28.811369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:28.811370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:28.811377 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:28.812471 (+ 1094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:28.812812 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:28.812813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:28.812814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:28.812864 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:28.813067 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:28.813068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:28.813069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:28.813173 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:28.813228 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:28.813470 (+ 242us) server_negotiation.cc:299] Negotiation successful
0924 07:16:28.813560 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 07:16:28.813802 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:28.805796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:42919)
0924 07:16:28.805989 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:28.806027 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:28.806527 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:28.806763 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:28.806767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:28.807350 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:28.807542 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:28.807548 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:28.808291 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:28.808292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:28.809075 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:28.809078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:28.809424 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:28.809425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:28.809541 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:28.810005 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:28.810014 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:28.811283 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:28.812540 (+ 1257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:28.812547 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:28.812549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:28.812752 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:28.812915 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:28.812916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:28.812916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:28.813017 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:28.813225 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:28.813232 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:28.813361 (+ 129us) client_negotiation.cc:769] Sending connection context
0924 07:16:28.813447 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 07:16:28.813591 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260924 07:16:28.814267 24664 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39257
I20260924 07:16:28.814344 24664 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:28.814425 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 24535
I20260924 07:16:28.814498 24664 heartbeater.cc:511] Master 127.10.130.254:39257 requested a full tablet report, sending...
I20260924 07:16:28.814525 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:16:28.814718 10763 external_mini_cluster.cc:1428] Setting key 7a83484955cbea9b1cc64edee57308ce
I20260924 07:16:28.815024 24473 ts_manager.cc:194] Registered new tserver with Master: 418086cdd66044d7b0fa1027e82dbf77 (127.10.130.193:40785)
I20260924 07:16:28.815603 24473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:42919
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:16:28.835932 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39257
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:28.913195 24669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:28.913378 24669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:28.913398 24669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:28.914496 24669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:28.914534 24669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:28.914561 24669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:16:28.915940 24669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:28.916168 24669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:28.916366 24669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:28.920862 24674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:28.920876 24675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:28.920864 24677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:28.920879 24669 server_base.cc:1034] running on GCE node
I20260924 07:16:28.921291 24669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:28.921495 24669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:28.922632 24669 hybrid_clock.cc:648] HybridClock initialized: now 1790234188922619 us; error 29 us; skew 500 ppm
Sep 24 07:16:28 dist-test-slave-7q7m krb5kdc[24428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234188, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:28.924930 24669 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:16:28.925297 24669 webserver.cc:460] Webserver started at http://127.10.130.194:33233/ using document root <none> and password file <none>
I20260924 07:16:28.925489 24669 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:28.925530 24669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:28.925648 24669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:16:28.926576 24669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "b3968a38e0de4a989795afd2903c1ef7"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "1064344ef613c82e44f68eaffb9303dc"
server_key_iv: "849cddda97db804163816363aefc7b87"
server_key_version: "encryptionkey@0"
I20260924 07:16:28.926882 24669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "b3968a38e0de4a989795afd2903c1ef7"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "1064344ef613c82e44f68eaffb9303dc"
server_key_iv: "849cddda97db804163816363aefc7b87"
server_key_version: "encryptionkey@0"
I20260924 07:16:28.928062 24669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:16:28.928678 24684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:28.928823 24669 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:16:28.928893 24669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "b3968a38e0de4a989795afd2903c1ef7"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "1064344ef613c82e44f68eaffb9303dc"
server_key_iv: "849cddda97db804163816363aefc7b87"
server_key_version: "encryptionkey@0"
I20260924 07:16:28.928956 24669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:28.954013 24669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:28.955756 24669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:28.955881 24669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:28.956120 24669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:28.956415 24669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:28.956449 24669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:28.956478 24669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:28.956501 24669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:28.961843 24669 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:46495
I20260924 07:16:28.961897 24797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:46495 every 8 connection(s)
I20260924 07:16:28.962188 24669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
Sep 24 07:16:28 dist-test-slave-7q7m krb5kdc[24428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234188, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:28.970165 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 24669
I20260924 07:16:28.970270 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:16:28.970449 10763 external_mini_cluster.cc:1428] Setting key 3a4e1e64dc39e2046edca485d1b929f6
I20260924 07:16:28.970470 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:28.962887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:36903 (local address 127.10.130.254:39257)
0924 07:16:28.962955 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:28.962957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:28.963094 (+ 137us) server_negotiation.cc:407] Connection header received
0924 07:16:28.963564 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:28.963565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:28.963575 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:28.963613 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:28.964525 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:28.965023 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:28.965966 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:28.966140 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:28.966181 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:28.968134 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:28.968136 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:28.968136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:28.968144 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:28.969351 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:28.969651 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:28.969651 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:28.969651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:28.969696 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:28.969872 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:28.969873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:28.969873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:28.969970 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:28.970041 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:28.970297 (+ 256us) server_negotiation.cc:299] Negotiation successful
0924 07:16:28.970380 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 07:16:28.970657 24800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:28.962845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:36903)
0924 07:16:28.962979 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:28.963016 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:28.963478 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:28.963671 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:28.963674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:28.964252 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:28.964448 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:28.964453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:28.965089 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:28.965089 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:28.965901 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:28.965902 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:28.966195 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:28.966196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:28.966303 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:28.966730 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:28.966738 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:28.968051 (+ 1313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:28.969411 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:28.969414 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:28.969415 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:28.969599 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:28.969746 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:28.969746 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:28.969746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:28.969833 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:28.970038 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:28.970041 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:28.970158 (+ 117us) client_negotiation.cc:769] Sending connection context
0924 07:16:28.970240 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 07:16:28.970390 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":74,"threads_started":1}
I20260924 07:16:28.971220 24798 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39257
I20260924 07:16:28.971306 24798 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:28.971470 24798 heartbeater.cc:511] Master 127.10.130.254:39257 requested a full tablet report, sending...
I20260924 07:16:28.971827 24473 ts_manager.cc:194] Registered new tserver with Master: b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495)
I20260924 07:16:28.972303 24473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:36903
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:16:28.992415 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39257
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:29.069795 24803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:29.069985 24803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:29.070006 24803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:29.071070 24803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:29.071101 24803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:29.071125 24803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:16:29.072468 24803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:29.072660 24803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:29.072856 24803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 07:16:29.077306 24803 server_base.cc:1034] running on GCE node
W20260924 07:16:29.077286 24808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:29.077276 24809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:29.077268 24811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:29.077637 24803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:29.077860 24803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:29.078992 24803 hybrid_clock.cc:648] HybridClock initialized: now 1790234189078982 us; error 24 us; skew 500 ppm
Sep 24 07:16:29 dist-test-slave-7q7m krb5kdc[24428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234189, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:29.081238 24803 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:16:29.081614 24803 webserver.cc:460] Webserver started at http://127.10.130.195:36401/ using document root <none> and password file <none>
I20260924 07:16:29.081784 24803 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:29.081820 24803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:29.081911 24803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:16:29.082823 24803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "8f33b636303d4b1e8be44df88262a8e1"
format_stamp: "Formatted at 2026-09-24 07:16:29 on dist-test-slave-7q7m"
server_key: "5b85ee6166389d3b78c9e185b0ef7cad"
server_key_iv: "96afe2d56f4a68c9b8dbe77d2bed6296"
server_key_version: "encryptionkey@0"
I20260924 07:16:29.083103 24803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "8f33b636303d4b1e8be44df88262a8e1"
format_stamp: "Formatted at 2026-09-24 07:16:29 on dist-test-slave-7q7m"
server_key: "5b85ee6166389d3b78c9e185b0ef7cad"
server_key_iv: "96afe2d56f4a68c9b8dbe77d2bed6296"
server_key_version: "encryptionkey@0"
I20260924 07:16:29.084283 24803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:16:29.084896 24818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:29.085034 24803 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:16:29.085112 24803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "8f33b636303d4b1e8be44df88262a8e1"
format_stamp: "Formatted at 2026-09-24 07:16:29 on dist-test-slave-7q7m"
server_key: "5b85ee6166389d3b78c9e185b0ef7cad"
server_key_iv: "96afe2d56f4a68c9b8dbe77d2bed6296"
server_key_version: "encryptionkey@0"
I20260924 07:16:29.085168 24803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:29.094447 24803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:29.096349 24803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:29.096472 24803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:29.096654 24803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:29.096910 24803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:29.096942 24803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:29.096967 24803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:29.096992 24803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:29.102347 24803 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:45165
I20260924 07:16:29.102399 24931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:45165 every 8 connection(s)
I20260924 07:16:29.102655 24803 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:16:29.106352 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 24803
I20260924 07:16:29.106446 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:16:29.106621 10763 external_mini_cluster.cc:1428] Setting key 71afc44b4c12b71152e3cbaf9ac55687
Sep 24 07:16:29 dist-test-slave-7q7m krb5kdc[24428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234189, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:29.111151 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.103344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:55797 (local address 127.10.130.254:39257)
0924 07:16:29.103401 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.103403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.103553 (+ 150us) server_negotiation.cc:407] Connection header received
0924 07:16:29.104052 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.104053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.104061 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.104118 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:29.104966 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.105463 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.106466 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.106760 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.106817 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.108842 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:29.108843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:29.108844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:29.108850 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:29.110090 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.110403 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.110403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.110404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.110449 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.110690 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.110690 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.110690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.110792 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:29.110836 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.111034 (+ 198us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.111089 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 07:16:29.111342 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.103299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:55797)
0924 07:16:29.103444 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.103482 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.103928 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.104174 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.104177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.104723 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:29.104904 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.104908 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.105547 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.105548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.106360 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.106362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.106838 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.106839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.106978 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.107373 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:29.107383 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:29.108764 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:29.110154 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:29.110158 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:29.110160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:29.110349 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:29.110520 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:29.110521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:29.110521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:29.110636 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:29.110835 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:29.110838 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:29.110943 (+ 105us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.111017 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.111137 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260924 07:16:29.111756 24932 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39257
I20260924 07:16:29.111835 24932 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:29.112027 24932 heartbeater.cc:511] Master 127.10.130.254:39257 requested a full tablet report, sending...
I20260924 07:16:29.112382 24473 ts_manager.cc:194] Registered new tserver with Master: 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165)
I20260924 07:16:29.112746 24473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:55797
I20260924 07:16:29.118327 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:16:29.124696 24512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.119519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36218 (local address 127.10.130.254:39257)
0924 07:16:29.119580 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.119583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.119675 (+ 92us) server_negotiation.cc:407] Connection header received
0924 07:16:29.119723 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.119724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.119732 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.119774 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:29.120294 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.120762 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.121426 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.121598 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.121652 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.122638 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:29.122639 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:29.122639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:29.122645 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:29.123798 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.124091 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.124093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.124093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.124150 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.124303 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.124303 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.124304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.124423 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:29.124492 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.124562 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.124626 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 07:16:29.126029 24473 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:16:29.127506 24473 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:16:29.133313 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.129432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:46495 (local address 127.0.0.1:37938)
0924 07:16:29.129766 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.129775 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.129816 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.130084 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.130085 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.130087 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:29.130123 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.130125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.131068 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.131069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.132404 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.132406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.133032 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.133033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.133116 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.133131 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.133195 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.133242 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":2,"thread_start_us":55,"threads_started":1}
I20260924 07:16:29.133432 24943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.129624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:45165 (local address 127.0.0.1:59374)
0924 07:16:29.129832 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.129840 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.129879 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.130124 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.130126 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.130128 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:29.130174 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.130175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.131444 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.131445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.132591 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.132592 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.133205 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.133206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.133263 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.133267 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.133319 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.133356 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":26,"thread_start_us":77,"threads_started":1}
I20260924 07:16:29.133692 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.129617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37938 (local address 127.10.130.194:46495)
0924 07:16:29.129777 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.129784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.129865 (+ 81us) server_negotiation.cc:407] Connection header received
0924 07:16:29.129908 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.129910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.129925 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.129997 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:29.130197 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.130985 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.132475 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.132961 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.133025 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.133082 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.133483 (+ 401us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.133525 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260924 07:16:29.133713 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.129818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59374 (local address 127.10.130.195:45165)
0924 07:16:29.129989 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.129992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.130001 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:16:29.130033 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.130035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.130047 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.130118 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:29.130224 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.131354 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.132658 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.133133 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.133207 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.133260 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.133333 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.133643 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":81,"threads_started":1}
I20260924 07:16:29.134168 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.129432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:40785 (local address 127.0.0.1:39596)
0924 07:16:29.129553 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.129596 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.129659 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.129998 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.130001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.130002 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:29.130062 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.130064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.131464 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.131465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.132409 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.132411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.133974 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.133976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.134028 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.134031 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.134074 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.134109 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":25,"thread_start_us":54,"threads_started":1}
I20260924 07:16:29.134223 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.129541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39596 (local address 127.10.130.193:40785)
0924 07:16:29.129839 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.129843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.129854 (+ 11us) server_negotiation.cc:407] Connection header received
0924 07:16:29.129895 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.129897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.129914 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.129999 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:29.130253 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.131366 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.133118 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.133830 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.133895 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.133944 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.134108 (+ 164us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.134150 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":92,"threads_started":1}
I20260924 07:16:29.134291 24732 tablet_service.cc:1467] Processing CreateTablet for tablet 2817083dea4546f7b716833d3ff6f21f (DEFAULT_TABLE table=test-table [id=756635a1574e49778213f5b71daa47d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:16:29.134652 24732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2817083dea4546f7b716833d3ff6f21f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:29.134831 24598 tablet_service.cc:1467] Processing CreateTablet for tablet 2817083dea4546f7b716833d3ff6f21f (DEFAULT_TABLE table=test-table [id=756635a1574e49778213f5b71daa47d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:16:29.135102 24598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2817083dea4546f7b716833d3ff6f21f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:29.136329 24866 tablet_service.cc:1467] Processing CreateTablet for tablet 2817083dea4546f7b716833d3ff6f21f (DEFAULT_TABLE table=test-table [id=756635a1574e49778213f5b71daa47d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:16:29.136551 24866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2817083dea4546f7b716833d3ff6f21f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:29.138005 24946 tablet_bootstrap.cc:492] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7: Bootstrap starting.
I20260924 07:16:29.138321 24947 tablet_bootstrap.cc:492] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Bootstrap starting.
I20260924 07:16:29.138697 24946 tablet_bootstrap.cc:654] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:29.138901 24947 tablet_bootstrap.cc:654] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:29.139230 24946 log.cc:824] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:29.139464 24948 tablet_bootstrap.cc:492] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1: Bootstrap starting.
I20260924 07:16:29.139603 24947 log.cc:824] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:29.140103 24948 tablet_bootstrap.cc:654] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:29.140316 24947 tablet_bootstrap.cc:492] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: No bootstrap required, opened a new log
I20260924 07:16:29.140383 24947 ts_tablet_manager.cc:1397] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:16:29.140610 24948 log.cc:824] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:29.141227 24946 tablet_bootstrap.cc:492] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7: No bootstrap required, opened a new log
I20260924 07:16:29.141280 24946 ts_tablet_manager.cc:1397] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 07:16:29.141916 24947 raft_consensus.cc:348] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } } peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } }
I20260924 07:16:29.142046 24947 raft_consensus.cc:374] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:29.142077 24947 raft_consensus.cc:729] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418086cdd66044d7b0fa1027e82dbf77, State: Initialized, Role: FOLLOWER
I20260924 07:16:29.142189 24947 consensus_queue.cc:260] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } } peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } }
I20260924 07:16:29.142432 24947 ts_tablet_manager.cc:1428] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:16:29.142439 24664 heartbeater.cc:503] Master 127.10.130.254:39257 was elected leader, sending a full tablet report...
I20260924 07:16:29.142746 24946 raft_consensus.cc:348] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } } peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } }
I20260924 07:16:29.142891 24946 raft_consensus.cc:374] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:29.142922 24946 raft_consensus.cc:729] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3968a38e0de4a989795afd2903c1ef7, State: Initialized, Role: FOLLOWER
I20260924 07:16:29.142997 24948 tablet_bootstrap.cc:492] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1: No bootstrap required, opened a new log
I20260924 07:16:29.143052 24948 ts_tablet_manager.cc:1397] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260924 07:16:29.143038 24946 consensus_queue.cc:260] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } } peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } }
I20260924 07:16:29.143277 24946 ts_tablet_manager.cc:1428] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:16:29.143393 24798 heartbeater.cc:503] Master 127.10.130.254:39257 was elected leader, sending a full tablet report...
I20260924 07:16:29.144690 24948 raft_consensus.cc:348] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } } peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } }
I20260924 07:16:29.144800 24948 raft_consensus.cc:374] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:29.144820 24948 raft_consensus.cc:729] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f33b636303d4b1e8be44df88262a8e1, State: Initialized, Role: FOLLOWER
I20260924 07:16:29.144917 24948 consensus_queue.cc:260] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } } peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } }
I20260924 07:16:29.145147 24948 ts_tablet_manager.cc:1428] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260924 07:16:29.145196 24932 heartbeater.cc:503] Master 127.10.130.254:39257 was elected leader, sending a full tablet report...
W20260924 07:16:29.212725 24799 tablet.cc:2369] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:16:29.277379 24954 raft_consensus.cc:482] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:16:29.277503 24954 raft_consensus.cc:504] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } } peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } }
I20260924 07:16:29.277817 24954 leader_election.cc:288] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 418086cdd66044d7b0fa1027e82dbf77 (127.10.130.193:40785), b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495)
I20260924 07:16:29.281443 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.278067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:57875 (local address 127.10.130.193:40785)
0924 07:16:29.278182 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.278184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.278191 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:16:29.278217 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.278218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.278227 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.278288 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:29.278404 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.279141 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.280551 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.281103 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.281176 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.281184 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.281336 (+ 152us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.281374 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260924 07:16:29.281443 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.278087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:49357 (local address 127.10.130.194:46495)
0924 07:16:29.278148 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.278149 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.278197 (+ 48us) server_negotiation.cc:407] Connection header received
0924 07:16:29.278244 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.278245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.278253 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.278299 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:29.278487 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.279182 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.280538 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.281119 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.281176 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.281184 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.281336 (+ 152us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.281373 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 07:16:29.281458 24934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.277973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:40785 (local address 127.10.130.195:57875)
0924 07:16:29.278014 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.278025 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.278068 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.278292 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.278292 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.278294 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:29.278354 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.278356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.279212 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.279213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.280462 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.280465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.281179 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.281180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.281276 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.281278 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.281326 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.281366 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":12}
I20260924 07:16:29.281466 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.277983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:46495 (local address 127.10.130.195:49357)
0924 07:16:29.278129 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.278140 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.278204 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.278404 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.278404 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.278405 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:29.278440 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.278441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.279248 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.279249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.280462 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.280465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.281185 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.281185 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.281272 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.281276 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.281326 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.281366 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":26,"thread_start_us":92,"threads_started":1}
I20260924 07:16:29.281795 24618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2817083dea4546f7b716833d3ff6f21f" candidate_uuid: "8f33b636303d4b1e8be44df88262a8e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "418086cdd66044d7b0fa1027e82dbf77" is_pre_election: true
I20260924 07:16:29.281958 24618 raft_consensus.cc:2454] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f33b636303d4b1e8be44df88262a8e1 in term 0.
I20260924 07:16:29.281998 24752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2817083dea4546f7b716833d3ff6f21f" candidate_uuid: "8f33b636303d4b1e8be44df88262a8e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3968a38e0de4a989795afd2903c1ef7" is_pre_election: true
I20260924 07:16:29.282156 24752 raft_consensus.cc:2454] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f33b636303d4b1e8be44df88262a8e1 in term 0.
I20260924 07:16:29.282152 24819 leader_election.cc:302] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 418086cdd66044d7b0fa1027e82dbf77, 8f33b636303d4b1e8be44df88262a8e1; no voters:
I20260924 07:16:29.282253 24954 raft_consensus.cc:2781] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:16:29.282316 24954 raft_consensus.cc:482] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:16:29.282338 24954 raft_consensus.cc:3037] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:29.283013 24954 raft_consensus.cc:504] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } } peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } }
I20260924 07:16:29.283116 24954 leader_election.cc:288] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [CANDIDATE]: Term 1 election: Requested vote from peers 418086cdd66044d7b0fa1027e82dbf77 (127.10.130.193:40785), b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495)
I20260924 07:16:29.283242 24618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2817083dea4546f7b716833d3ff6f21f" candidate_uuid: "8f33b636303d4b1e8be44df88262a8e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "418086cdd66044d7b0fa1027e82dbf77"
I20260924 07:16:29.283278 24752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2817083dea4546f7b716833d3ff6f21f" candidate_uuid: "8f33b636303d4b1e8be44df88262a8e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3968a38e0de4a989795afd2903c1ef7"
I20260924 07:16:29.283316 24618 raft_consensus.cc:3037] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:29.283363 24752 raft_consensus.cc:3037] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:29.284034 24752 raft_consensus.cc:2454] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f33b636303d4b1e8be44df88262a8e1 in term 1.
I20260924 07:16:29.284044 24618 raft_consensus.cc:2454] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f33b636303d4b1e8be44df88262a8e1 in term 1.
I20260924 07:16:29.284191 24819 leader_election.cc:302] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 418086cdd66044d7b0fa1027e82dbf77, 8f33b636303d4b1e8be44df88262a8e1; no voters:
I20260924 07:16:29.284287 24954 raft_consensus.cc:2781] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:16:29.284427 24954 raft_consensus.cc:686] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [term 1 LEADER]: Becoming Leader. State: Replica: 8f33b636303d4b1e8be44df88262a8e1, State: Running, Role: LEADER
I20260924 07:16:29.284514 24954 consensus_queue.cc:237] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } } peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } }
I20260924 07:16:29.285266 24472 catalog_manager.cc:5495] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195). New cstate: current_term: 1 leader_uuid: "8f33b636303d4b1e8be44df88262a8e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } health_report { overall_health: UNKNOWN } } }
W20260924 07:16:29.305738 24665 tablet.cc:2369] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:16:29.311826 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.309836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59390 (local address 127.10.130.195:45165)
0924 07:16:29.309884 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.309886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.309902 (+ 16us) server_negotiation.cc:407] Connection header received
0924 07:16:29.309933 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.309934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.309950 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.310015 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:16:29.310132 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.310585 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.311266 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.311441 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.311482 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.311577 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:16:29.311668 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:16:29.311726 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.311755 (+ 29us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.311771 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:16:29.313526 24618 raft_consensus.cc:1261] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Refusing update from remote peer 8f33b636303d4b1e8be44df88262a8e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:16:29.313535 24752 raft_consensus.cc:1261] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7 [term 1 FOLLOWER]: Refusing update from remote peer 8f33b636303d4b1e8be44df88262a8e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:16:29.313859 24954 consensus_queue.cc:1035] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:16:29.314003 24956 consensus_queue.cc:1035] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:16:29.315924 24959 mvcc.cc:205] Tried to move back new op lower bound from 7332799239425744896 to 7332799239309996032. Current Snapshot: MvccSnapshot[applied={T|T < 7332799239425744896}]
I20260924 07:16:29.316298 24961 mvcc.cc:205] Tried to move back new op lower bound from 7332799239425744896 to 7332799239309996032. Current Snapshot: MvccSnapshot[applied={T|T < 7332799239425744896}]
I20260924 07:16:29.317868 24960 mvcc.cc:205] Tried to move back new op lower bound from 7332799239425744896 to 7332799239309996032. Current Snapshot: MvccSnapshot[applied={T|T < 7332799239425744896 or (T in {7332799239425744896})}]
I20260924 07:16:29.318125 24472 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:16:29.318199 24472 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:16:29.318874 24472 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Sending DeleteTablet for 3 replicas of tablet 2817083dea4546f7b716833d3ff6f21f
I20260924 07:16:29.319206 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 24535
I20260924 07:16:29.319195 24866 tablet_service.cc:1514] Processing DeleteTablet for tablet 2817083dea4546f7b716833d3ff6f21f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:29 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:59374
I20260924 07:16:29.319201 24732 tablet_service.cc:1514] Processing DeleteTablet for tablet 2817083dea4546f7b716833d3ff6f21f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:29 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:37938
I20260924 07:16:29.319893 24969 tablet_replica.cc:330] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1: stopping tablet replica
I20260924 07:16:29.319959 24969 raft_consensus.cc:2229] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:16:29.320056 24969 raft_consensus.cc:2258] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:16:29.320075 24970 tablet_replica.cc:330] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7: stopping tablet replica
I20260924 07:16:29.320152 24970 raft_consensus.cc:2229] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:16:29.320237 24970 raft_consensus.cc:2258] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:16:29.320495 24969 ts_tablet_manager.cc:1905] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:16:29.320566 24970 ts_tablet_manager.cc:1905] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:16:29.321992 24969 ts_tablet_manager.cc:1918] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:16:29.322000 24970 ts_tablet_manager.cc:1918] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:16:29.322052 24970 log.cc:1196] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/2817083dea4546f7b716833d3ff6f21f
I20260924 07:16:29.322052 24969 log.cc:1196] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/2817083dea4546f7b716833d3ff6f21f
I20260924 07:16:29.322299 24969 ts_tablet_manager.cc:1939] T 2817083dea4546f7b716833d3ff6f21f P 8f33b636303d4b1e8be44df88262a8e1: Deleting consensus metadata
I20260924 07:16:29.322504 24970 ts_tablet_manager.cc:1939] T 2817083dea4546f7b716833d3ff6f21f P b3968a38e0de4a989795afd2903c1ef7: Deleting consensus metadata
I20260924 07:16:29.322659 24457 catalog_manager.cc:4841] TS 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165): tablet 2817083dea4546f7b716833d3ff6f21f (table test-table [id=756635a1574e49778213f5b71daa47d5]) successfully deleted
I20260924 07:16:29.322949 24458 catalog_manager.cc:4841] TS b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495): tablet 2817083dea4546f7b716833d3ff6f21f (table test-table [id=756635a1574e49778213f5b71daa47d5]) successfully deleted
W20260924 07:16:29.326172 24457 connection.cc:535] client connection to 127.10.130.193:40785 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 07:16:29.326257 24457 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 07:16:29.326372 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 24669
I20260924 07:16:29.327311 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.327195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:40785 (local address 127.0.0.1:39598)
0924 07:16:29.327245 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.327284 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 07:16:29.327422 24457 catalog_manager.cc:4584] TS 418086cdd66044d7b0fa1027e82dbf77 (127.10.130.193:40785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2817083dea4546f7b716833d3ff6f21f: Network error: Client connection negotiation failed: client connection to 127.10.130.193:40785: connect: Connection refused (error 111)
I20260924 07:16:29.331272 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 24803
I20260924 07:16:29.335278 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 24441
I20260924 07:16:29.339250 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39257
--webserver_interface=127.10.130.254
--webserver_port=39407
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:39257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:29.417171 24972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:29.417363 24972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:29.417392 24972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:29.418493 24972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:16:29.418531 24972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:29.418550 24972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:29.418565 24972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:16:29.418581 24972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:16:29.420150 24972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:39257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=39407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:29.420379 24972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:29.420575 24972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:29.428284 24981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:29.428284 24978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:29.428375 24972 server_base.cc:1034] running on GCE node
W20260924 07:16:29.428505 24979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:29.428766 24972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:29.429065 24972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:29.430234 24972 hybrid_clock.cc:648] HybridClock initialized: now 1790234189430220 us; error 27 us; skew 500 ppm
Sep 24 07:16:29 dist-test-slave-7q7m krb5kdc[24428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234189, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:29.432772 24972 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:16:29.433238 24972 webserver.cc:460] Webserver started at http://127.10.130.254:39407/ using document root <none> and password file <none>
I20260924 07:16:29.433471 24972 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:29.433543 24972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:29.434906 24972 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 07:16:29.435472 24988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:29.435629 24972 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:16:29.435720 24972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "59a88f46ff3c47b285844a5c4e454849"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "a5a5892fa36e19c5d8bf86c8688704d3"
server_key_iv: "67ff218427a8ace435436f529b98a9da"
server_key_version: "encryptionkey@0"
I20260924 07:16:29.436143 24972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:29.451712 24972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:29.453440 24972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:29.453547 24972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:29.457159 24972 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:39257
I20260924 07:16:29.457204 25040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:39257 every 8 connection(s)
I20260924 07:16:29.457564 24972 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:16:29.459105 25041 sys_catalog.cc:263] Verifying existing consensus state
I20260924 07:16:29.459673 25041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Bootstrap starting.
I20260924 07:16:29.463406 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 24972
I20260924 07:16:29.463663 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:40785
--local_ip_for_outbound_sockets=127.10.130.193
--tserver_master_addrs=127.10.130.254:39257
--webserver_port=41365
--webserver_interface=127.10.130.193
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:16:29.467801 25041 log.cc:824] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:29.469636 25041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 07:16:29.469897 25041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Bootstrap complete.
I20260924 07:16:29.471925 25041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } }
I20260924 07:16:29.472206 25041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59a88f46ff3c47b285844a5c4e454849, State: Initialized, Role: FOLLOWER
I20260924 07:16:29.472363 25041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } }
I20260924 07:16:29.472434 25041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:16:29.472484 25041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:16:29.472528 25041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 1 FOLLOWER]: Advancing to term 2
I20260924 07:16:29.473534 25041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } }
I20260924 07:16:29.473625 25041 leader_election.cc:302] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59a88f46ff3c47b285844a5c4e454849; no voters:
I20260924 07:16:29.473765 25041 leader_election.cc:288] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 07:16:29.473812 25045 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 07:16:29.473927 25045 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 2 LEADER]: Becoming Leader. State: Replica: 59a88f46ff3c47b285844a5c4e454849, State: Running, Role: LEADER
I20260924 07:16:29.474005 25041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:16:29.474011 25045 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } }
I20260924 07:16:29.474395 25047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "59a88f46ff3c47b285844a5c4e454849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } } }
I20260924 07:16:29.474468 25047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:29.474776 25046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59a88f46ff3c47b285844a5c4e454849. Latest consensus state: current_term: 2 leader_uuid: "59a88f46ff3c47b285844a5c4e454849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } } }
I20260924 07:16:29.474839 25046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:29.474921 25050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:16:29.475584 25050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:16:29.475672 25050 catalog_manager.cc:1259] Loaded cluster ID: 93c58b2222a84c519ab04d7b100b196d
I20260924 07:16:29.475708 25050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:16:29.476176 25050 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260924 07:16:29.480757 25050 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260924 07:16:29.483037 25050 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:16:29.483254 25050 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Loaded TSK: 0
I20260924 07:16:29.483479 25050 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 07:16:29.543907 25043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:29.544150 25043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:29.544183 25043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:29.545331 25043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:29.545370 25043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:29.545406 25043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:16:29.546887 25043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:40785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=41365
--tserver_master_addrs=127.10.130.254:39257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:29.547111 25043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:29.547308 25043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:29.551895 25068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:29.551895 25071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:29.552161 25069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:29.552428 25043 server_base.cc:1034] running on GCE node
I20260924 07:16:29.552641 25043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:29.552914 25043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:29.554096 25043 hybrid_clock.cc:648] HybridClock initialized: now 1790234189554081 us; error 36 us; skew 500 ppm
Sep 24 07:16:29 dist-test-slave-7q7m krb5kdc[24428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234189, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:29.557034 25043 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:16:29.557581 25043 webserver.cc:460] Webserver started at http://127.10.130.193:41365/ using document root <none> and password file <none>
I20260924 07:16:29.557842 25043 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:29.557920 25043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:29.559605 25043 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:16:29.560348 25078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:29.560513 25043 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:29.560592 25043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "418086cdd66044d7b0fa1027e82dbf77"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "50a962637fe1c0b136ec64f4cf5922e4"
server_key_iv: "d26f93a42ce3ed480bd4cbfec6baf25f"
server_key_version: "encryptionkey@0"
I20260924 07:16:29.560921 25043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:29.576905 25043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:29.578977 25043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:29.579087 25043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:29.579303 25043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:29.579736 25085 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 07:16:29.581606 25043 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 07:16:29.581651 25043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 07:16:29.581686 25043 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 07:16:29.582283 25043 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 07:16:29.582315 25043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 07:16:29.582408 25085 tablet_bootstrap.cc:492] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Bootstrap starting.
I20260924 07:16:29.588760 25043 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:40785
I20260924 07:16:29.588886 25192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:40785 every 8 connection(s)
I20260924 07:16:29.589160 25043 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:16:29.593060 25085 log.cc:824] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:29.595793 25085 tablet_bootstrap.cc:492] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 07:16:29.596104 25085 tablet_bootstrap.cc:492] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Bootstrap complete.
I20260924 07:16:29.597216 25085 ts_tablet_manager.cc:1397] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20260924 07:16:29.598749 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 25043
I20260924 07:16:29.598531 25085 raft_consensus.cc:348] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } } peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } }
I20260924 07:16:29.598834 25085 raft_consensus.cc:729] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418086cdd66044d7b0fa1027e82dbf77, State: Initialized, Role: FOLLOWER
I20260924 07:16:29.598985 25085 consensus_queue.cc:260] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } } peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } }
I20260924 07:16:29.599032 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:46495
--local_ip_for_outbound_sockets=127.10.130.194
--tserver_master_addrs=127.10.130.254:39257
--webserver_port=33233
--webserver_interface=127.10.130.194
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:16:29.599234 25085 ts_tablet_manager.cc:1428] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
Sep 24 07:16:29 dist-test-slave-7q7m krb5kdc[24428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234189, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:29.603510 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.589797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:33161)
0924 07:16:29.589959 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.590002 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.590441 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.593725 (+ 3284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.593729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.594959 (+ 1230us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:29.595153 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.595159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.596372 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.596373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.597161 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.597163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.597776 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.597778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.597901 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.598374 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:29.598382 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:29.599958 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:29.601824 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:29.601831 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:29.601833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:29.602109 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:29.602369 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:29.602369 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:29.602370 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:29.602480 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:29.602920 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:29.602923 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:29.603064 (+ 141us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.603163 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.603312 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260924 07:16:29.603902 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.589893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:33161 (local address 127.10.130.254:39257)
0924 07:16:29.593270 (+ 3377us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.593280 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.593330 (+ 50us) server_negotiation.cc:407] Connection header received
0924 07:16:29.593473 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.593480 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.593539 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.593689 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:29.595277 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.596291 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.597247 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.597670 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.597773 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.600077 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:29.600081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:29.600081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:29.600093 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:29.601728 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.602191 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.602194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.602195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.602306 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.602540 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.602541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.602541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.602850 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:29.602939 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.603242 (+ 303us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.603381 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3325,"thread_start_us":107,"threads_started":1}
I20260924 07:16:29.604784 25193 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39257
I20260924 07:16:29.604877 25193 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:29.605032 25193 heartbeater.cc:511] Master 127.10.130.254:39257 requested a full tablet report, sending...
I20260924 07:16:29.605772 25005 ts_manager.cc:194] Registered new tserver with Master: 418086cdd66044d7b0fa1027e82dbf77 (127.10.130.193:40785)
I20260924 07:16:29.606926 25005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:33161
I20260924 07:16:29.607815 25193 heartbeater.cc:503] Master 127.10.130.254:39257 was elected leader, sending a full tablet report...
Sep 24 07:16:29 dist-test-slave-7q7m krb5kdc[24428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234189, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for kudu/127.10.130.193@KRBTEST.COM
I20260924 07:16:29.614318 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.606616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39602 (local address 127.10.130.193:40785)
0924 07:16:29.606964 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.606968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.606977 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:16:29.607003 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.607005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.607017 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.607070 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:29.607762 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.608689 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.609747 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.610017 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.610064 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.611904 (+ 1840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:29.611911 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:29.611911 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:29.611918 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:29.613293 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.613671 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.613673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.613673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.613720 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.613876 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.613877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.613877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.613985 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:29.614038 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.614189 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.614253 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":63,"threads_started":1}
I20260924 07:16:29.614367 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.606392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:40785 (local address 127.0.0.1:39602)
0924 07:16:29.606753 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.606787 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.606844 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.607125 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.607130 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.607623 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:29.607676 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.607679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.608763 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.608765 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.609676 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.609679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.610074 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.610074 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.610145 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.610545 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:29.610549 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:29.611829 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:29.613364 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:29.613370 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:29.613372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:29.613603 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:29.613770 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:29.613770 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:29.613771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:29.613829 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:29.614032 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:29.614037 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:29.614083 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.614159 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.614272 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":74,"threads_started":1}
I20260924 07:16:29.614760 25127 tablet_service.cc:1514] Processing DeleteTablet for tablet 2817083dea4546f7b716833d3ff6f21f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:29 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:39602
I20260924 07:16:29.615345 25202 tablet_replica.cc:330] stopping tablet replica
I20260924 07:16:29.615435 25202 raft_consensus.cc:2229] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:16:29.615497 25202 raft_consensus.cc:2258] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:16:29.615741 25202 ts_tablet_manager.cc:1905] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:16:29.617559 25202 ts_tablet_manager.cc:1918] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:16:29.617621 25202 log.cc:1196] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal/wals/2817083dea4546f7b716833d3ff6f21f
I20260924 07:16:29.617830 25202 ts_tablet_manager.cc:1939] T 2817083dea4546f7b716833d3ff6f21f P 418086cdd66044d7b0fa1027e82dbf77: Deleting consensus metadata
I20260924 07:16:29.618263 24989 catalog_manager.cc:4841] TS 418086cdd66044d7b0fa1027e82dbf77 (127.10.130.193:40785): tablet 2817083dea4546f7b716833d3ff6f21f (table test-table [id=756635a1574e49778213f5b71daa47d5]) successfully deleted
W20260924 07:16:29.680918 25199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:29.681111 25199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:29.681133 25199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:29.682291 25199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:29.682329 25199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:29.682355 25199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:16:29.683806 25199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:46495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=33233
--tserver_master_addrs=127.10.130.254:39257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:29.684056 25199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:29.684274 25199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:29.688722 25207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:29.688715 25208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:29.688714 25210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:29.688925 25199 server_base.cc:1034] running on GCE node
I20260924 07:16:29.689102 25199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:29.689302 25199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:29.690428 25199 hybrid_clock.cc:648] HybridClock initialized: now 1790234189690417 us; error 24 us; skew 500 ppm
Sep 24 07:16:29 dist-test-slave-7q7m krb5kdc[24428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234189, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:29.696105 25199 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:16:29.696528 25199 webserver.cc:460] Webserver started at http://127.10.130.194:33233/ using document root <none> and password file <none>
I20260924 07:16:29.696740 25199 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:29.696799 25199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:29.697983 25199 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:16:29.698490 25217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:29.698627 25199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:29.698709 25199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "b3968a38e0de4a989795afd2903c1ef7"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "1064344ef613c82e44f68eaffb9303dc"
server_key_iv: "849cddda97db804163816363aefc7b87"
server_key_version: "encryptionkey@0"
I20260924 07:16:29.699005 25199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:29.717796 25199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:29.719592 25199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:29.719735 25199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:29.719960 25199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:29.720301 25199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:29.720335 25199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:29.720363 25199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:29.720386 25199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:29.725660 25199 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:46495
I20260924 07:16:29.725700 25330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:46495 every 8 connection(s)
I20260924 07:16:29.726015 25199 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
Sep 24 07:16:29 dist-test-slave-7q7m krb5kdc[24428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234189, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:29.733820 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.726658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:33311 (local address 127.10.130.254:39257)
0924 07:16:29.726703 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.726705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.726890 (+ 185us) server_negotiation.cc:407] Connection header received
0924 07:16:29.727357 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.727358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.727368 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.727416 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:29.728312 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.728791 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.729688 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.729876 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.729929 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.731731 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:29.731732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:29.731733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:29.731739 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:29.732831 (+ 1092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.733154 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.733155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.733155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.733200 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.733370 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.733371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.733371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.733461 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:29.733502 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.733695 (+ 193us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.733758 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:16:29.733982 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.726636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:33311)
0924 07:16:29.726781 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.726815 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.727278 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.727470 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.727473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.728038 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:29.728248 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.728254 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.728850 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.728851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.729630 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.729631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.729932 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.729932 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.730023 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.730388 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:29.730396 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:29.731643 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:29.732899 (+ 1256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:29.732904 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:29.732906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:29.733100 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:29.733244 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:29.733244 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:29.733245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:29.733330 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:29.733499 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:29.733502 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:29.733604 (+ 102us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.733685 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.733794 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
I20260924 07:16:29.734278 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 25199
I20260924 07:16:29.734453 25331 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39257
I20260924 07:16:29.734542 25331 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:29.734563 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:45165
--local_ip_for_outbound_sockets=127.10.130.195
--tserver_master_addrs=127.10.130.254:39257
--webserver_port=36401
--webserver_interface=127.10.130.195
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:16:29.734720 25331 heartbeater.cc:511] Master 127.10.130.254:39257 requested a full tablet report, sending...
I20260924 07:16:29.735095 25005 ts_manager.cc:194] Registered new tserver with Master: b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495)
I20260924 07:16:29.735440 25005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:33311
W20260924 07:16:29.813212 25334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:29.813408 25334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:29.813431 25334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:29.814524 25334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:29.814555 25334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:29.814580 25334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:16:29.815946 25334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:45165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=36401
--tserver_master_addrs=127.10.130.254:39257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:29.816159 25334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:29.816339 25334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:29.820876 25340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:29.820886 25339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:29.821046 25342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:29.821059 25334 server_base.cc:1034] running on GCE node
I20260924 07:16:29.821295 25334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:29.821506 25334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:29.822636 25334 hybrid_clock.cc:648] HybridClock initialized: now 1790234189822625 us; error 23 us; skew 500 ppm
Sep 24 07:16:29 dist-test-slave-7q7m krb5kdc[24428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234189, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:29.824887 25334 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:16:29.825279 25334 webserver.cc:460] Webserver started at http://127.10.130.195:36401/ using document root <none> and password file <none>
I20260924 07:16:29.825470 25334 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:29.825531 25334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:29.826781 25334 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:16:29.827281 25349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:29.827438 25334 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:16:29.827514 25334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "8f33b636303d4b1e8be44df88262a8e1"
format_stamp: "Formatted at 2026-09-24 07:16:29 on dist-test-slave-7q7m"
server_key: "5b85ee6166389d3b78c9e185b0ef7cad"
server_key_iv: "96afe2d56f4a68c9b8dbe77d2bed6296"
server_key_version: "encryptionkey@0"
I20260924 07:16:29.827821 25334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:29.839608 25334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:29.841688 25334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:29.841820 25334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:29.842015 25334 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:29.842288 25334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:29.842319 25334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:29.842347 25334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:29.842371 25334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:29.847674 25334 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:45165
I20260924 07:16:29.847710 25462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:45165 every 8 connection(s)
I20260924 07:16:29.848054 25334 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:16:29.848869 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 25334
Sep 24 07:16:29 dist-test-slave-7q7m krb5kdc[24428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234189, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:29.854810 25467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.849204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36222 (local address 127.10.130.254:39257)
0924 07:16:29.849302 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.849303 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.849311 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:16:29.849375 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.849376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.849387 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.849441 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:29.849939 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.850438 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.851129 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.851334 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.851386 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.852505 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:29.852506 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:29.852507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:29.852512 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:29.853716 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.854031 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.854032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.854033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.854101 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.854286 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.854287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.854287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.854387 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:29.854537 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.854599 (+ 62us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.854713 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":52,"threads_started":1}
I20260924 07:16:29.856635 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.848688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:52501 (local address 127.10.130.254:39257)
0924 07:16:29.848757 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.848759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.848956 (+ 197us) server_negotiation.cc:407] Connection header received
0924 07:16:29.849500 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.849501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.849514 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.849582 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:29.850439 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.850911 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.851839 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.852082 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.852131 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.854212 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:29.854213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:29.854214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:29.854220 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:29.855632 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.855948 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.855949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.855949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.856023 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:29.856205 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:29.856205 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:29.856206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:29.856292 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:29.856334 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.856532 (+ 198us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.856581 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 07:16:29.856835 25465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.848685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:52501)
0924 07:16:29.848827 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.848866 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.849412 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.849654 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.849658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.850179 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:29.850355 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.850359 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.850990 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.850991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.851762 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.851764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.852158 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.852159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.852279 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.852675 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:29.852684 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:29.854133 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:29.855700 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:29.855705 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:29.855707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:29.855898 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:29.856072 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:29.856072 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:29.856073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:29.856161 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:29.856335 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:29.856338 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:29.856441 (+ 103us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.856514 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.856628 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260924 07:16:29.857234 25463 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39257
I20260924 07:16:29.857313 25463 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:29.857446 25463 heartbeater.cc:511] Master 127.10.130.254:39257 requested a full tablet report, sending...
I20260924 07:16:29.857761 25005 ts_manager.cc:194] Registered new tserver with Master: 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165)
I20260924 07:16:29.858068 25005 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:52501
I20260924 07:16:29.866235 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:16:29.868674 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.866555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36230 (local address 127.10.130.254:39257)
0924 07:16:29.866592 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.866594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.866682 (+ 88us) server_negotiation.cc:407] Connection header received
0924 07:16:29.866743 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.866743 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.866773 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.866827 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:16:29.866914 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.867363 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.868060 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.868236 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.868281 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.868448 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:16:29.868531 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:16:29.868581 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.868614 (+ 33us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.868627 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:16:29.868912 25005 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:16:29.870452 25005 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:16:29.872419 25127 tablet_service.cc:1467] Processing CreateTablet for tablet 09ade8b4e298496e8b9d8aa57a7eee70 (DEFAULT_TABLE table=test-table [id=12a5941ef7684337a53449141edd9436]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:16:29.872613 25127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09ade8b4e298496e8b9d8aa57a7eee70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:29.874246 25085 tablet_bootstrap.cc:492] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77: Bootstrap starting.
I20260924 07:16:29.874655 25085 tablet_bootstrap.cc:654] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:29.875293 25085 tablet_bootstrap.cc:492] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77: No bootstrap required, opened a new log
I20260924 07:16:29.875335 25085 ts_tablet_manager.cc:1397] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 07:16:29.875665 25085 raft_consensus.cc:348] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:29.875748 25085 raft_consensus.cc:374] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:29.875790 25085 raft_consensus.cc:729] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418086cdd66044d7b0fa1027e82dbf77, State: Initialized, Role: FOLLOWER
I20260924 07:16:29.875795 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.872034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:45165 (local address 127.0.0.1:59396)
0924 07:16:29.872131 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.872140 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.872183 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.872632 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.872634 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.872636 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:29.872690 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.872691 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.873669 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.873670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.874676 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.874679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.875529 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.875530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.875605 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.875625 (+ 20us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.875694 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.875737 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260924 07:16:29.875818 25470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.872125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59396 (local address 127.10.130.195:45165)
0924 07:16:29.872469 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.872472 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.872480 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:16:29.872510 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.872512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.872527 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.872764 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:29.872827 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.873584 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.874759 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.875448 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.875517 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.875566 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.875706 (+ 140us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.875744 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":85,"threads_started":1}
I20260924 07:16:29.875833 25085 consensus_queue.cc:260] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:29.875922 25085 ts_tablet_manager.cc:1428] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260924 07:16:29.876130 25469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.872057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:46495 (local address 127.0.0.1:37944)
0924 07:16:29.872210 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.872221 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.872274 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.872651 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.872652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.872653 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:29.872691 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.872692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.874068 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.874069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.875074 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.875077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.875913 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.875914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.876000 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.876005 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.876033 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.876068 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260924 07:16:29.876417 25471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.872314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37944 (local address 127.10.130.194:46495)
0924 07:16:29.872471 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.872475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.872485 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:16:29.872520 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.872522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.872537 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.872653 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:29.872751 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.873955 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.875164 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.875819 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.876221 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.876279 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.876304 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.876343 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260924 07:16:29.876456 25397 tablet_service.cc:1467] Processing CreateTablet for tablet 09ade8b4e298496e8b9d8aa57a7eee70 (DEFAULT_TABLE table=test-table [id=12a5941ef7684337a53449141edd9436]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:16:29.876722 25397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09ade8b4e298496e8b9d8aa57a7eee70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:29.876969 25265 tablet_service.cc:1467] Processing CreateTablet for tablet 09ade8b4e298496e8b9d8aa57a7eee70 (DEFAULT_TABLE table=test-table [id=12a5941ef7684337a53449141edd9436]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:16:29.877256 25265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09ade8b4e298496e8b9d8aa57a7eee70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:29.879184 25473 tablet_bootstrap.cc:492] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1: Bootstrap starting.
I20260924 07:16:29.879781 25474 tablet_bootstrap.cc:492] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7: Bootstrap starting.
I20260924 07:16:29.879927 25473 tablet_bootstrap.cc:654] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:29.880328 25474 tablet_bootstrap.cc:654] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:29.880386 25473 log.cc:824] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:29.880719 25474 log.cc:824] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:29.881057 25473 tablet_bootstrap.cc:492] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1: No bootstrap required, opened a new log
I20260924 07:16:29.881129 25473 ts_tablet_manager.cc:1397] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:16:29.881379 25474 tablet_bootstrap.cc:492] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7: No bootstrap required, opened a new log
I20260924 07:16:29.881434 25474 ts_tablet_manager.cc:1397] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:16:29.882431 25473 raft_consensus.cc:348] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:29.882571 25473 raft_consensus.cc:374] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:29.882606 25473 raft_consensus.cc:729] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f33b636303d4b1e8be44df88262a8e1, State: Initialized, Role: FOLLOWER
I20260924 07:16:29.882750 25473 consensus_queue.cc:260] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:29.882858 25474 raft_consensus.cc:348] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:29.882975 25473 ts_tablet_manager.cc:1428] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:16:29.883020 25463 heartbeater.cc:503] Master 127.10.130.254:39257 was elected leader, sending a full tablet report...
I20260924 07:16:29.883015 25474 raft_consensus.cc:374] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:29.883051 25474 raft_consensus.cc:729] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3968a38e0de4a989795afd2903c1ef7, State: Initialized, Role: FOLLOWER
I20260924 07:16:29.883174 25474 consensus_queue.cc:260] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:29.883423 25474 ts_tablet_manager.cc:1428] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 07:16:29.883451 25331 heartbeater.cc:503] Master 127.10.130.254:39257 was elected leader, sending a full tablet report...
I20260924 07:16:29.897260 25198 raft_consensus.cc:482] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:16:29.897367 25198 raft_consensus.cc:504] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:29.897656 25198 leader_election.cc:288] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165), b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495)
I20260924 07:16:29.900417 25471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.897895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:55367 (local address 127.10.130.194:46495)
0924 07:16:29.897929 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.897931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.897984 (+ 53us) server_negotiation.cc:407] Connection header received
0924 07:16:29.898044 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.898044 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.898053 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.898102 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:29.898239 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.898666 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.899624 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.900076 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.900142 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.900152 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.900305 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.900348 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:16:29.900436 25470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.897814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:43765 (local address 127.10.130.195:45165)
0924 07:16:29.897856 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.897858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.897869 (+ 11us) server_negotiation.cc:407] Connection header received
0924 07:16:29.897971 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.897971 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.897981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.898033 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:29.898170 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.898645 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.899702 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.900102 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.900170 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.900177 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.900347 (+ 170us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.900383 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 07:16:29.900424 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.897813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:46495 (local address 127.10.130.193:55367)
0924 07:16:29.897934 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.897943 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.897997 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.898110 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.898111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.898113 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:29.898182 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.898184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.898736 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.898737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.899521 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.899523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.900160 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.900161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.900223 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.900245 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.900286 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.900320 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":79,"threads_started":1}
I20260924 07:16:29.900439 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.897779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:45165 (local address 127.10.130.193:43765)
0924 07:16:29.897827 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.897838 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:29.897889 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:29.898042 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:29.898042 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:29.898044 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:29.898113 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.898115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.898713 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.898714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.899617 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:29.899620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.900178 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.900179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:29.900268 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.900271 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:16:29.900325 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 07:16:29.900364 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":21}
I20260924 07:16:29.900817 25285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09ade8b4e298496e8b9d8aa57a7eee70" candidate_uuid: "418086cdd66044d7b0fa1027e82dbf77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3968a38e0de4a989795afd2903c1ef7" is_pre_election: true
I20260924 07:16:29.900923 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09ade8b4e298496e8b9d8aa57a7eee70" candidate_uuid: "418086cdd66044d7b0fa1027e82dbf77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f33b636303d4b1e8be44df88262a8e1" is_pre_election: true
I20260924 07:16:29.900971 25285 raft_consensus.cc:2454] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 418086cdd66044d7b0fa1027e82dbf77 in term 0.
I20260924 07:16:29.901051 25417 raft_consensus.cc:2454] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 418086cdd66044d7b0fa1027e82dbf77 in term 0.
I20260924 07:16:29.901182 25080 leader_election.cc:302] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 418086cdd66044d7b0fa1027e82dbf77, b3968a38e0de4a989795afd2903c1ef7; no voters:
I20260924 07:16:29.901304 25198 raft_consensus.cc:2781] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:16:29.901367 25198 raft_consensus.cc:482] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:16:29.901391 25198 raft_consensus.cc:3037] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:29.902108 25198 raft_consensus.cc:504] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:29.902225 25198 leader_election.cc:288] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 1 election: Requested vote from peers 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165), b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495)
I20260924 07:16:29.902376 25285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09ade8b4e298496e8b9d8aa57a7eee70" candidate_uuid: "418086cdd66044d7b0fa1027e82dbf77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3968a38e0de4a989795afd2903c1ef7"
I20260924 07:16:29.902385 25417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09ade8b4e298496e8b9d8aa57a7eee70" candidate_uuid: "418086cdd66044d7b0fa1027e82dbf77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f33b636303d4b1e8be44df88262a8e1"
I20260924 07:16:29.902438 25285 raft_consensus.cc:3037] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:29.902489 25417 raft_consensus.cc:3037] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:29.903127 25285 raft_consensus.cc:2454] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 418086cdd66044d7b0fa1027e82dbf77 in term 1.
I20260924 07:16:29.903220 25417 raft_consensus.cc:2454] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 418086cdd66044d7b0fa1027e82dbf77 in term 1.
I20260924 07:16:29.903275 25080 leader_election.cc:302] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 418086cdd66044d7b0fa1027e82dbf77, b3968a38e0de4a989795afd2903c1ef7; no voters:
I20260924 07:16:29.903355 25198 raft_consensus.cc:2781] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:16:29.903515 25198 raft_consensus.cc:686] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 LEADER]: Becoming Leader. State: Replica: 418086cdd66044d7b0fa1027e82dbf77, State: Running, Role: LEADER
I20260924 07:16:29.903590 25198 consensus_queue.cc:237] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:29.904351 25004 catalog_manager.cc:5495] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 418086cdd66044d7b0fa1027e82dbf77 (127.10.130.193). New cstate: current_term: 1 leader_uuid: "418086cdd66044d7b0fa1027e82dbf77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } health_report { overall_health: HEALTHY } } }
I20260924 07:16:29.913096 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.911211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39608 (local address 127.10.130.193:40785)
0924 07:16:29.911249 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:29.911251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:29.911258 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:16:29.911315 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:29.911315 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:29.911329 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:29.911382 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:16:29.911452 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.911887 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.912551 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:29.912731 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:29.912774 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:29.912869 (+ 95us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:16:29.912951 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:16:29.912994 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:29.913022 (+ 28us) server_negotiation.cc:299] Negotiation successful
0924 07:16:29.913038 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:16:29.914317 25285 raft_consensus.cc:1261] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7 [term 1 FOLLOWER]: Refusing update from remote peer 418086cdd66044d7b0fa1027e82dbf77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:16:29.914319 25417 raft_consensus.cc:1261] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1 [term 1 FOLLOWER]: Refusing update from remote peer 418086cdd66044d7b0fa1027e82dbf77: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:16:29.914582 25198 consensus_queue.cc:1035] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:16:29.914707 25480 consensus_queue.cc:1035] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:16:29.917150 25484 mvcc.cc:205] Tried to move back new op lower bound from 7332799241887256576 to 7332799241845768192. Current Snapshot: MvccSnapshot[applied={T|T < 7332799241887256576 or (T in {7332799241887256576})}]
I20260924 07:16:29.917773 25485 mvcc.cc:205] Tried to move back new op lower bound from 7332799241887256576 to 7332799241845768192. Current Snapshot: MvccSnapshot[applied={T|T < 7332799241887256576}]
I20260924 07:16:29.918946 25004 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:16:29.919026 25004 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:16:29.919715 25004 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Sending DeleteTablet for 3 replicas of tablet 09ade8b4e298496e8b9d8aa57a7eee70
I20260924 07:16:29.920060 25265 tablet_service.cc:1514] Processing DeleteTablet for tablet 09ade8b4e298496e8b9d8aa57a7eee70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:29 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:37944
I20260924 07:16:29.920079 25397 tablet_service.cc:1514] Processing DeleteTablet for tablet 09ade8b4e298496e8b9d8aa57a7eee70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:29 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:59396
I20260924 07:16:29.920251 25493 tablet_replica.cc:330] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7: stopping tablet replica
I20260924 07:16:29.920252 25127 tablet_service.cc:1514] Processing DeleteTablet for tablet 09ade8b4e298496e8b9d8aa57a7eee70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:29 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:39602
I20260924 07:16:29.920281 25494 tablet_replica.cc:330] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1: stopping tablet replica
I20260924 07:16:29.920327 25202 tablet_replica.cc:330] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77: stopping tablet replica
I20260924 07:16:29.920339 25493 raft_consensus.cc:2229] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:16:29.920351 25494 raft_consensus.cc:2229] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:16:29.920368 25202 raft_consensus.cc:2229] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:16:29.920428 25493 raft_consensus.cc:2258] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:16:29.920428 25494 raft_consensus.cc:2258] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:16:29.920431 25202 raft_consensus.cc:2258] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:16:29.920584 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 25043
I20260924 07:16:29.921384 25494 ts_tablet_manager.cc:1905] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:16:29.921602 25493 ts_tablet_manager.cc:1905] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:16:29.922927 25494 ts_tablet_manager.cc:1918] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:16:29.922953 25493 ts_tablet_manager.cc:1918] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:16:29.923004 25493 log.cc:1196] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/09ade8b4e298496e8b9d8aa57a7eee70
I20260924 07:16:29.923004 25494 log.cc:1196] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/09ade8b4e298496e8b9d8aa57a7eee70
I20260924 07:16:29.923271 25493 ts_tablet_manager.cc:1939] T 09ade8b4e298496e8b9d8aa57a7eee70 P b3968a38e0de4a989795afd2903c1ef7: Deleting consensus metadata
I20260924 07:16:29.923271 25494 ts_tablet_manager.cc:1939] T 09ade8b4e298496e8b9d8aa57a7eee70 P 8f33b636303d4b1e8be44df88262a8e1: Deleting consensus metadata
I20260924 07:16:29.923619 24990 catalog_manager.cc:4841] TS b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495): tablet 09ade8b4e298496e8b9d8aa57a7eee70 (table test-table [id=12a5941ef7684337a53449141edd9436]) successfully deleted
I20260924 07:16:29.923758 24989 catalog_manager.cc:4841] TS 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165): tablet 09ade8b4e298496e8b9d8aa57a7eee70 (table test-table [id=12a5941ef7684337a53449141edd9436]) successfully deleted
W20260924 07:16:29.925763 24989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:40785 (error 108)
I20260924 07:16:29.925992 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 25199
I20260924 07:16:29.926465 25469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:29.926066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:40785 (local address 127.0.0.1:39622)
0924 07:16:29.926402 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:29.926434 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:40785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322}
W20260924 07:16:29.926573 24989 catalog_manager.cc:4584] TS 418086cdd66044d7b0fa1027e82dbf77 (127.10.130.193:40785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 09ade8b4e298496e8b9d8aa57a7eee70: Network error: Client connection negotiation failed: client connection to 127.10.130.193:40785: connect: Connection refused (error 111)
I20260924 07:16:29.930048 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 25334
I20260924 07:16:29.933669 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 24972
I20260924 07:16:29.937893 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39257
--webserver_interface=127.10.130.254
--webserver_port=39407
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:39257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:30.014716 25496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:30.014905 25496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:30.014925 25496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:30.016039 25496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:16:30.016075 25496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:30.016088 25496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:30.016111 25496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:16:30.016134 25496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:16:30.017529 25496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:39257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=39407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:30.017755 25496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:30.017952 25496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:30.022421 25504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:30.022431 25502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:30.022437 25501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:30.022634 25496 server_base.cc:1034] running on GCE node
I20260924 07:16:30.022915 25496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:30.023195 25496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:30.024328 25496 hybrid_clock.cc:648] HybridClock initialized: now 1790234190024316 us; error 24 us; skew 500 ppm
Sep 24 07:16:30 dist-test-slave-7q7m krb5kdc[24428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234190, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:30.026499 25496 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:16:30.026885 25496 webserver.cc:460] Webserver started at http://127.10.130.254:39407/ using document root <none> and password file <none>
I20260924 07:16:30.027086 25496 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:30.027133 25496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:30.028445 25496 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:16:30.029047 25511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:30.029196 25496 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:16:30.029273 25496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "59a88f46ff3c47b285844a5c4e454849"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "a5a5892fa36e19c5d8bf86c8688704d3"
server_key_iv: "67ff218427a8ace435436f529b98a9da"
server_key_version: "encryptionkey@0"
I20260924 07:16:30.029574 25496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:30.039672 25496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:30.041270 25496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:30.041391 25496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:30.044950 25496 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:39257
I20260924 07:16:30.044982 25563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:39257 every 8 connection(s)
I20260924 07:16:30.045305 25496 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:16:30.046717 25564 sys_catalog.cc:263] Verifying existing consensus state
I20260924 07:16:30.047277 25564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Bootstrap starting.
I20260924 07:16:30.051805 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 25496
I20260924 07:16:30.052148 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:40785
--local_ip_for_outbound_sockets=127.10.130.193
--tserver_master_addrs=127.10.130.254:39257
--webserver_port=41365
--webserver_interface=127.10.130.193
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:16:30.055322 25564 log.cc:824] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:30.057376 25564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 07:16:30.057621 25564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849: Bootstrap complete.
I20260924 07:16:30.059559 25564 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } }
I20260924 07:16:30.059764 25564 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59a88f46ff3c47b285844a5c4e454849, State: Initialized, Role: FOLLOWER
I20260924 07:16:30.059896 25564 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } }
I20260924 07:16:30.059957 25564 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:16:30.060038 25564 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:16:30.060089 25564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 2 FOLLOWER]: Advancing to term 3
I20260924 07:16:30.060982 25564 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } }
I20260924 07:16:30.061077 25564 leader_election.cc:302] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59a88f46ff3c47b285844a5c4e454849; no voters:
I20260924 07:16:30.061218 25564 leader_election.cc:288] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 07:16:30.061261 25568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 07:16:30.061363 25568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [term 3 LEADER]: Becoming Leader. State: Replica: 59a88f46ff3c47b285844a5c4e454849, State: Running, Role: LEADER
I20260924 07:16:30.061458 25564 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:16:30.061450 25568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } }
I20260924 07:16:30.061723 25569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "59a88f46ff3c47b285844a5c4e454849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } } }
I20260924 07:16:30.061805 25569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:30.062017 25570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59a88f46ff3c47b285844a5c4e454849. Latest consensus state: current_term: 3 leader_uuid: "59a88f46ff3c47b285844a5c4e454849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59a88f46ff3c47b285844a5c4e454849" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39257 } } }
I20260924 07:16:30.062111 25570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59a88f46ff3c47b285844a5c4e454849 [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:30.062233 25573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:16:30.062997 25573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:16:30.063103 25573 catalog_manager.cc:1259] Loaded cluster ID: 93c58b2222a84c519ab04d7b100b196d
I20260924 07:16:30.063148 25573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260924 07:16:30.063354 25573 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5cae5289ad google::LogMessage::Fail() at ??:0
@ 0x7f5cae52b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f5cae5284de google::LogMessage::Flush() at ??:0
@ 0x7f5cae529a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5cb64490f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f5cb64528b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5cb03f60d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5cb03eb954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5cb124f6db start_thread at ??:0
@ 0x7f5cad02661f clone at ??:0
W20260924 07:16:30.143414 25566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:30.143591 25566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:30.143612 25566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:30.144753 25566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:30.144791 25566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:30.144816 25566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:16:30.146186 25566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:40785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=41365
--tserver_master_addrs=127.10.130.254:39257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:30.146370 25566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:30.146553 25566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:30.151289 25603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:30.151295 25600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:30.151315 25601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:30.151834 25566 server_base.cc:1034] running on GCE node
I20260924 07:16:30.152067 25566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:30.152331 25566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:30.153507 25566 hybrid_clock.cc:648] HybridClock initialized: now 1790234190153488 us; error 32 us; skew 500 ppm
Sep 24 07:16:30 dist-test-slave-7q7m krb5kdc[24428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234190, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:30.156380 25566 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:16:30.156898 25566 webserver.cc:460] Webserver started at http://127.10.130.193:41365/ using document root <none> and password file <none>
I20260924 07:16:30.157117 25566 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:30.157171 25566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:30.158690 25566 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:16:30.159387 25610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:30.159550 25566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:16:30.159629 25566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "418086cdd66044d7b0fa1027e82dbf77"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "50a962637fe1c0b136ec64f4cf5922e4"
server_key_iv: "d26f93a42ce3ed480bd4cbfec6baf25f"
server_key_version: "encryptionkey@0"
I20260924 07:16:30.159950 25566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:30.171155 25566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:30.173185 25566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:30.173300 25566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:30.173497 25566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:30.173902 25617 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 07:16:30.175731 25566 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 07:16:30.175774 25566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 07:16:30.175805 25566 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 07:16:30.176424 25566 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 07:16:30.176455 25566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 07:16:30.176540 25617 tablet_bootstrap.cc:492] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77: Bootstrap starting.
I20260924 07:16:30.182796 25566 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:40785
I20260924 07:16:30.182881 25724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:40785 every 8 connection(s)
I20260924 07:16:30.183182 25566 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:16:30.186987 25617 log.cc:824] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:30.187109 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 25566
I20260924 07:16:30.187361 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:46495
--local_ip_for_outbound_sockets=127.10.130.194
--tserver_master_addrs=127.10.130.254:39257
--webserver_port=33233
--webserver_interface=127.10.130.194
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:16:30.189478 25617 tablet_bootstrap.cc:492] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 07:16:30.189761 25617 tablet_bootstrap.cc:492] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77: Bootstrap complete.
I20260924 07:16:30.190827 25617 ts_tablet_manager.cc:1397] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.006s
I20260924 07:16:30.192210 25617 raft_consensus.cc:348] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:30.192521 25617 raft_consensus.cc:729] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 418086cdd66044d7b0fa1027e82dbf77, State: Initialized, Role: FOLLOWER
I20260924 07:16:30.192687 25617 consensus_queue.cc:260] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:30.192941 25617 ts_tablet_manager.cc:1428] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260924 07:16:30.261006 25727 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:16:30.183750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:54877)
0924 07:16:30.183903 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:30.183943 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:30.184396 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:30.260840 (+ 76444us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
W20260924 07:16:30.261269 25725 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:39257 (0 consecutive failures): Network error: Failed to ping master at 127.10.130.254:39257: Client connection negotiation failed: client connection to 127.10.130.254:39257: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 07:16:30.261610 25727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:30.261500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:33135)
0924 07:16:30.261556 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:30.261589 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260924 07:16:30.261915 25727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:30.261837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:44961)
0924 07:16:30.261878 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:30.261899 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 07:16:30.262028 25725 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 07:16:30.267201 25729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:30.267381 25729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:30.267402 25729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:30.268514 25729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:30.268551 25729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:30.268575 25729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:16:30.269928 25729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:46495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=33233
--tserver_master_addrs=127.10.130.254:39257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:30.270100 25729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:30.270265 25729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:30.274778 25735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:30.274858 25729 server_base.cc:1034] running on GCE node
W20260924 07:16:30.274958 25738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:30.274777 25736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:30.275200 25729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:30.275427 25729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:30.276571 25729 hybrid_clock.cc:648] HybridClock initialized: now 1790234190276558 us; error 29 us; skew 500 ppm
Sep 24 07:16:30 dist-test-slave-7q7m krb5kdc[24428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234190, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:30.278764 25729 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:16:30.279138 25729 webserver.cc:460] Webserver started at http://127.10.130.194:33233/ using document root <none> and password file <none>
I20260924 07:16:30.279318 25729 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:30.279364 25729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:30.280586 25729 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260924 07:16:30.281057 25745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:30.281205 25729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:30.281279 25729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "b3968a38e0de4a989795afd2903c1ef7"
format_stamp: "Formatted at 2026-09-24 07:16:28 on dist-test-slave-7q7m"
server_key: "1064344ef613c82e44f68eaffb9303dc"
server_key_iv: "849cddda97db804163816363aefc7b87"
server_key_version: "encryptionkey@0"
I20260924 07:16:30.281575 25729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:30.300927 25729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:30.302611 25729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:30.302749 25729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:30.302950 25729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:30.303238 25729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:30.303269 25729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:30.303298 25729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:30.303320 25729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:30.308532 25729 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:46495
I20260924 07:16:30.308591 25858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:46495 every 8 connection(s)
I20260924 07:16:30.308892 25729 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:16:30.309829 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:30.309485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:55751)
0924 07:16:30.309617 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:30.309694 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
W20260924 07:16:30.310168 25859 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:39257 (0 consecutive failures): Network error: Failed to ping master at 127.10.130.254:39257: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
I20260924 07:16:30.310400 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:30.310325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:43547)
0924 07:16:30.310360 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:30.310383 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260924 07:16:30.310607 25861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:30.310549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:41049)
0924 07:16:30.310578 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:30.310595 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260924 07:16:30.310681 25859 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 07:16:30.312393 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 25729
I20260924 07:16:30.312638 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:45165
--local_ip_for_outbound_sockets=127.10.130.195
--tserver_master_addrs=127.10.130.254:39257
--webserver_port=36401
--webserver_interface=127.10.130.195
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:30.390488 25862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:30.390666 25862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:30.390686 25862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:30.391769 25862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:30.391808 25862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:30.391832 25862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:16:30.393234 25862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:45165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=36401
--tserver_master_addrs=127.10.130.254:39257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:30.393441 25862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:30.393635 25862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:30.398157 25870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:30.398211 25868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:30.398159 25867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:30.398182 25862 server_base.cc:1034] running on GCE node
I20260924 07:16:30.398530 25862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:30.398762 25862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:30.399899 25862 hybrid_clock.cc:648] HybridClock initialized: now 1790234190399884 us; error 28 us; skew 500 ppm
Sep 24 07:16:30 dist-test-slave-7q7m krb5kdc[24428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234190, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:30.402225 25862 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:16:30.402587 25862 webserver.cc:460] Webserver started at http://127.10.130.195:36401/ using document root <none> and password file <none>
I20260924 07:16:30.402786 25862 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:30.402827 25862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:30.404008 25862 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:16:30.404495 25877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:30.404630 25862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:30.404698 25862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "8f33b636303d4b1e8be44df88262a8e1"
format_stamp: "Formatted at 2026-09-24 07:16:29 on dist-test-slave-7q7m"
server_key: "5b85ee6166389d3b78c9e185b0ef7cad"
server_key_iv: "96afe2d56f4a68c9b8dbe77d2bed6296"
server_key_version: "encryptionkey@0"
I20260924 07:16:30.405031 25862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:30.417022 25862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:30.418689 25862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:30.418802 25862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:30.418992 25862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:30.419283 25862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:30.419315 25862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:30.419337 25862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:30.419359 25862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:30.426523 25862 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:45165
I20260924 07:16:30.426581 25990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:45165 every 8 connection(s)
I20260924 07:16:30.426882 25862 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:16:30.427810 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:30.427468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:40457)
0924 07:16:30.427599 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:30.427677 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
W20260924 07:16:30.428197 25991 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:39257 (0 consecutive failures): Network error: Failed to ping master at 127.10.130.254:39257: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
I20260924 07:16:30.428424 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:30.428348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:48765)
0924 07:16:30.428384 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:30.428408 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260924 07:16:30.428609 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:30.428565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:35975)
0924 07:16:30.428583 (+ 18us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:30.428599 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260924 07:16:30.428691 25991 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 07:16:30.436875 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 25862
I20260924 07:16:31.262570 25995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:31.262349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:39947)
0924 07:16:31.262473 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:31.262505 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260924 07:16:31.311097 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:31.310928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:51343)
0924 07:16:31.311022 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:31.311049 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":53,"threads_started":1}
I20260924 07:16:31.429203 25997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:31.429002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:54379)
0924 07:16:31.429120 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:31.429155 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
I20260924 07:16:31.706015 25998 raft_consensus.cc:482] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:16:31.706156 25998 raft_consensus.cc:504] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:31.706449 25998 leader_election.cc:288] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165), b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495)
Sep 24 07:16:31 dist-test-slave-7q7m krb5kdc[24428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234190, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.195@KRBTEST.COM
Sep 24 07:16:31 dist-test-slave-7q7m krb5kdc[24428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234190, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.194@KRBTEST.COM
I20260924 07:16:31.716138 26000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:31.706697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:34031 (local address 127.10.130.195:45165)
0924 07:16:31.706829 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:31.706841 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:31.706869 (+ 28us) server_negotiation.cc:407] Connection header received
0924 07:16:31.707285 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:31.707287 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:31.707326 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:31.707447 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:31.708204 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:31.709036 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:31.710154 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:31.710540 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:31.710640 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:31.712901 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:31.712905 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:31.712906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:31.712918 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:31.714645 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:31.715039 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:31.715042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:31.715042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:31.715135 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:31.715354 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:31.715355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:31.715355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:31.715566 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:31.715633 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:31.715893 (+ 260us) server_negotiation.cc:299] Negotiation successful
0924 07:16:31.716021 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
I20260924 07:16:31.716127 25995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:31.706564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:45165 (local address 127.10.130.193:34031)
0924 07:16:31.706609 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:31.706619 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:31.706668 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:31.707496 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:31.707499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:31.707885 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:31.708090 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:31.708095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:31.709137 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:31.709138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:31.710076 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:31.710079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:31.710621 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:31.710622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:31.710731 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:31.711243 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:31.711250 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:31.712777 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:31.714724 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:31.714730 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:31.714732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:31.714968 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:31.715193 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:31.715194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:31.715194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:31.715300 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:31.715627 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:31.715633 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:31.715753 (+ 120us) client_negotiation.cc:769] Sending connection context
0924 07:16:31.715835 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 07:16:31.716020 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":55}
I20260924 07:16:31.716140 26001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:31.706734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:34885 (local address 127.10.130.194:46495)
0924 07:16:31.706872 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:31.706880 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:31.706904 (+ 24us) server_negotiation.cc:407] Connection header received
0924 07:16:31.707308 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:31.707311 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:31.707346 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:31.707467 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:31.708208 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:31.709034 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:31.710154 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:31.710539 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:31.710633 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:31.713227 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:31.713231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:31.713231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:31.713244 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:31.714705 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:31.715052 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:31.715055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:31.715055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:31.715135 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:31.715350 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:31.715351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:31.715351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:31.715568 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:31.715634 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:31.715892 (+ 258us) server_negotiation.cc:299] Negotiation successful
0924 07:16:31.716035 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":50,"threads_started":1}
I20260924 07:16:31.716197 25999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:31.706609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:46495 (local address 127.10.130.193:34885)
0924 07:16:31.706740 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:31.706748 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:31.706781 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:31.707496 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:31.707499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:31.707896 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:31.708089 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:31.708095 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:31.709124 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:31.709125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:31.710077 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:31.710079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:31.710620 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:31.710621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:31.710730 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:31.711287 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:31.711291 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:31.713123 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:31.714778 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:31.714779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:31.714779 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:31.714988 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:31.715190 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:31.715191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:31.715191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:31.715298 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:31.715627 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:31.715632 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:31.715753 (+ 121us) client_negotiation.cc:769] Sending connection context
0924 07:16:31.715834 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 07:16:31.716079 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":170,"thread_start_us":65,"threads_started":1}
W20260924 07:16:31.716913 25612 leader_election.cc:341] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495): Not found: Tablet not found: 09ade8b4e298496e8b9d8aa57a7eee70
W20260924 07:16:31.717286 25611 leader_election.cc:341] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165): Not found: Tablet not found: 09ade8b4e298496e8b9d8aa57a7eee70
I20260924 07:16:31.717345 25611 leader_election.cc:302] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 418086cdd66044d7b0fa1027e82dbf77; no voters: 8f33b636303d4b1e8be44df88262a8e1, b3968a38e0de4a989795afd2903c1ef7
I20260924 07:16:31.717444 25998 raft_consensus.cc:2726] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 07:16:32.263176 26002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:32.262966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:53775)
0924 07:16:32.263091 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:32.263129 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260924 07:16:32.311635 26003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:32.311463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:60739)
0924 07:16:32.311568 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:32.311599 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":56,"threads_started":1}
I20260924 07:16:32.429731 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:32.429566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:37931)
0924 07:16:32.429664 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:32.429695 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":54,"threads_started":1}
I20260924 07:16:33.263818 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:33.263583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:52609)
0924 07:16:33.263728 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:33.263772 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260924 07:16:33.312251 26006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:33.312062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:44153)
0924 07:16:33.312178 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:33.312212 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
I20260924 07:16:33.430348 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:33.430151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:42881)
0924 07:16:33.430276 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:33.430310 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
I20260924 07:16:33.528617 26008 raft_consensus.cc:482] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:16:33.528712 26008 raft_consensus.cc:504] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:33.528877 26008 leader_election.cc:288] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165), b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495)
W20260924 07:16:33.529258 25611 leader_election.cc:341] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165): Not found: Tablet not found: 09ade8b4e298496e8b9d8aa57a7eee70
W20260924 07:16:33.529317 25612 leader_election.cc:341] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495): Not found: Tablet not found: 09ade8b4e298496e8b9d8aa57a7eee70
I20260924 07:16:33.529382 25612 leader_election.cc:302] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 418086cdd66044d7b0fa1027e82dbf77; no voters: 8f33b636303d4b1e8be44df88262a8e1, b3968a38e0de4a989795afd2903c1ef7
I20260924 07:16:33.529459 26008 raft_consensus.cc:2726] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 07:16:34.264428 26009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:34.264242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:60771)
0924 07:16:34.264347 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:34.264380 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260924 07:16:34.312840 26010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:34.312668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:44591)
0924 07:16:34.312777 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:34.312805 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":53,"threads_started":1}
I20260924 07:16:34.430891 26011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:34.430722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:35653)
0924 07:16:34.430819 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:34.430850 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":51,"threads_started":1}
I20260924 07:16:35.265058 26012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:35.264843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:42435)
0924 07:16:35.264965 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:35.265007 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260924 07:16:35.313483 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:35.313286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:53669)
0924 07:16:35.313402 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:35.313445 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
I20260924 07:16:35.431411 26014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:35.431224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:57423)
0924 07:16:35.431340 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:35.431374 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":61,"threads_started":1}
I20260924 07:16:36.265640 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:36.265443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:33881)
0924 07:16:36.265567 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:36.265595 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
I20260924 07:16:36.308696 26016 raft_consensus.cc:482] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:16:36.308784 26016 raft_consensus.cc:504] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:36.308938 26016 leader_election.cc:288] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165), b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495)
W20260924 07:16:36.309280 25611 leader_election.cc:341] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165): Not found: Tablet not found: 09ade8b4e298496e8b9d8aa57a7eee70
W20260924 07:16:36.309432 25612 leader_election.cc:341] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495): Not found: Tablet not found: 09ade8b4e298496e8b9d8aa57a7eee70
I20260924 07:16:36.309497 25612 leader_election.cc:302] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 418086cdd66044d7b0fa1027e82dbf77; no voters: 8f33b636303d4b1e8be44df88262a8e1, b3968a38e0de4a989795afd2903c1ef7
I20260924 07:16:36.309576 26016 raft_consensus.cc:2726] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 07:16:36.314083 26017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:36.313877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:48859)
0924 07:16:36.314002 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:36.314035 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260924 07:16:36.431957 26018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:36.431779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:50377)
0924 07:16:36.431887 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:36.431918 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":52,"threads_started":1}
I20260924 07:16:37.266245 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:37.266047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:59333)
0924 07:16:37.266164 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:37.266201 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260924 07:16:37.314677 26020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:37.314456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:48257)
0924 07:16:37.314584 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:37.314626 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260924 07:16:37.432552 26021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:37.432360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:48113)
0924 07:16:37.432475 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:37.432510 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260924 07:16:38.266844 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:38.266645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:48499)
0924 07:16:38.266767 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:38.266797 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260924 07:16:38.315304 26023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:38.315094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:37183)
0924 07:16:38.315212 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:38.315250 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260924 07:16:38.433105 26024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:38.432929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:33571)
0924 07:16:38.433038 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:38.433067 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260924 07:16:38.569798 26025 raft_consensus.cc:482] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:16:38.569888 26025 raft_consensus.cc:504] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:38.570053 26025 leader_election.cc:288] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165), b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495)
W20260924 07:16:38.570395 25611 leader_election.cc:341] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165): Not found: Tablet not found: 09ade8b4e298496e8b9d8aa57a7eee70
W20260924 07:16:38.570478 25612 leader_election.cc:341] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495): Not found: Tablet not found: 09ade8b4e298496e8b9d8aa57a7eee70
I20260924 07:16:38.570549 25612 leader_election.cc:302] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 418086cdd66044d7b0fa1027e82dbf77; no voters: 8f33b636303d4b1e8be44df88262a8e1, b3968a38e0de4a989795afd2903c1ef7
I20260924 07:16:38.570628 26025 raft_consensus.cc:2726] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 07:16:39.267395 26026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:39.267202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:38949)
0924 07:16:39.267318 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:39.267350 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260924 07:16:39.315864 26027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:39.315686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:42717)
0924 07:16:39.315790 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:39.315822 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":55,"threads_started":1}
I20260924 07:16:39.433660 26028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:39.433467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:53399)
0924 07:16:39.433590 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:39.433622 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260924 07:16:40.268093 26029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:40.267826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:47225)
0924 07:16:40.267963 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:40.268042 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":58,"threads_started":1}
I20260924 07:16:40.316499 26030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:40.316267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:49905)
0924 07:16:40.316401 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:40.316444 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":73,"threads_started":1}
I20260924 07:16:40.434181 26031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:40.433994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:56417)
0924 07:16:40.434109 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:40.434141 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260924 07:16:41.268673 26032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:41.268470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:35753)
0924 07:16:41.268596 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:41.268629 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260924 07:16:41.317036 26033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:41.316845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:34867)
0924 07:16:41.316961 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:41.316993 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260924 07:16:41.434755 26034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:41.434540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:46975)
0924 07:16:41.434669 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:41.434706 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260924 07:16:42.128885 26035 raft_consensus.cc:482] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:16:42.128984 26035 raft_consensus.cc:504] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f33b636303d4b1e8be44df88262a8e1" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 45165 } } peers { permanent_uuid: "b3968a38e0de4a989795afd2903c1ef7" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46495 } } peers { permanent_uuid: "418086cdd66044d7b0fa1027e82dbf77" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 40785 } }
I20260924 07:16:42.129156 26035 leader_election.cc:288] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165), b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495)
W20260924 07:16:42.129555 25611 leader_election.cc:341] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f33b636303d4b1e8be44df88262a8e1 (127.10.130.195:45165): Not found: Tablet not found: 09ade8b4e298496e8b9d8aa57a7eee70
W20260924 07:16:42.129642 25612 leader_election.cc:341] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b3968a38e0de4a989795afd2903c1ef7 (127.10.130.194:46495): Not found: Tablet not found: 09ade8b4e298496e8b9d8aa57a7eee70
I20260924 07:16:42.129683 25612 leader_election.cc:302] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 418086cdd66044d7b0fa1027e82dbf77; no voters: 8f33b636303d4b1e8be44df88262a8e1, b3968a38e0de4a989795afd2903c1ef7
I20260924 07:16:42.129755 26035 raft_consensus.cc:2726] T 09ade8b4e298496e8b9d8aa57a7eee70 P 418086cdd66044d7b0fa1027e82dbf77 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 07:16:42.269306 26036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:42.269069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:57705)
0924 07:16:42.269216 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:42.269259 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":62,"threads_started":1}
I20260924 07:16:42.317611 26037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:42.317396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:54861)
0924 07:16:42.317522 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:42.317569 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260924 07:16:42.435315 26038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:42.435124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:39099)
0924 07:16:42.435241 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:42.435274 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":69,"threads_started":1}
I20260924 07:16:43.269874 26039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:43.269682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:38647)
0924 07:16:43.269799 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:43.269830 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260924 07:16:43.318156 26040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:43.317957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:56659)
0924 07:16:43.318087 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:43.318120 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260924 07:16:43.435874 26041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:43.435676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:37645)
0924 07:16:43.435796 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:43.435830 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260924 07:16:44.270522 26042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:44.270271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:51293)
0924 07:16:44.270407 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:44.270449 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":60,"threads_started":1}
I20260924 07:16:44.318706 26043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:44.318500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:50187)
0924 07:16:44.318628 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:44.318661 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260924 07:16:44.436419 26044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:44.436220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:60913)
0924 07:16:44.436344 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:44.436376 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260924 07:16:45.271168 26045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:45.270973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.193:41565)
0924 07:16:45.271087 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:45.271124 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260924 07:16:45.319308 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:45.319108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.194:36061)
0924 07:16:45.319237 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:45.319268 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260924 07:16:45.437002 26047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:45.436790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39257 (local address 127.10.130.195:55989)
0924 07:16:45.436908 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:45.436962 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
I20260924 07:16:45.446148 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 25566
I20260924 07:16:45.450209 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 25729
I20260924 07:16:45.454038 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 25862
2026-09-24T07:16:45Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (17990 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:16:45 dist-test-slave-7q7m krb5kdc[26050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:16:45 dist-test-slave-7q7m krb5kdc[26050](info): set up 2 sockets
Sep 24 07:16:45 dist-test-slave-7q7m krb5kdc[26050](info): commencing operation
krb5kdc: starting...
W20260924 07:16:46.407648 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.935s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:16:46 dist-test-slave-7q7m krb5kdc[26050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:16:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:16:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:16:46.512532 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39723
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:39723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:46.589515 26063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:46.589717 26063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:46.589737 26063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:46.590803 26063 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 07:16:46.590834 26063 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 07:16:46.590847 26063 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 07:16:46.590857 26063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:16:46.590878 26063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:46.590889 26063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:46.590911 26063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:16:46.590922 26063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:16:46.592293 26063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:39723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:46.592470 26063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:46.592633 26063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:46.597106 26069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:46.597149 26068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:46.597180 26071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:46.597275 26063 server_base.cc:1034] running on GCE node
I20260924 07:16:46.597469 26063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:46.597693 26063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:46.598814 26063 hybrid_clock.cc:648] HybridClock initialized: now 1790234206598805 us; error 26 us; skew 500 ppm
Sep 24 07:16:46 dist-test-slave-7q7m krb5kdc[26050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234206, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:46.601092 26063 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:16:46.601491 26063 webserver.cc:460] Webserver started at http://127.10.130.254:36237/ using document root <none> and password file <none>
I20260924 07:16:46.601670 26063 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:46.601715 26063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:46.601831 26063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:16:46.602788 26063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "743d90099c2543348ccde444ddb60ad4"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "9e6b7e14ef3d2466720c75e24edd42c4"
server_key_iv: "646e2e70cca402a27631d7a94712bf53"
server_key_version: "encryptionkey@0"
I20260924 07:16:46.603149 26063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "743d90099c2543348ccde444ddb60ad4"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "9e6b7e14ef3d2466720c75e24edd42c4"
server_key_iv: "646e2e70cca402a27631d7a94712bf53"
server_key_version: "encryptionkey@0"
I20260924 07:16:46.604326 26063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:16:46.604918 26078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:46.605062 26063 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:16:46.605141 26063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "743d90099c2543348ccde444ddb60ad4"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "9e6b7e14ef3d2466720c75e24edd42c4"
server_key_iv: "646e2e70cca402a27631d7a94712bf53"
server_key_version: "encryptionkey@0"
I20260924 07:16:46.605198 26063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:46.622603 26063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:46.624532 26063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:46.624660 26063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:46.628208 26063 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:39723
I20260924 07:16:46.628237 26130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:39723 every 8 connection(s)
I20260924 07:16:46.628538 26063 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:16:46.629268 26131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:46.631349 26131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Bootstrap starting.
I20260924 07:16:46.631855 26131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:46.632133 26131 log.cc:824] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:46.632767 26131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: No bootstrap required, opened a new log
I20260924 07:16:46.634241 26131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } }
I20260924 07:16:46.634371 26131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:46.634397 26131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 743d90099c2543348ccde444ddb60ad4, State: Initialized, Role: FOLLOWER
I20260924 07:16:46.634500 26131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } }
I20260924 07:16:46.634548 26131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:16:46.634567 26131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:16:46.634596 26131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:46.635036 26131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } }
I20260924 07:16:46.635115 26131 leader_election.cc:302] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 743d90099c2543348ccde444ddb60ad4; no voters:
I20260924 07:16:46.635228 26131 leader_election.cc:288] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:16:46.635272 26134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:16:46.635427 26134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 1 LEADER]: Becoming Leader. State: Replica: 743d90099c2543348ccde444ddb60ad4, State: Running, Role: LEADER
I20260924 07:16:46.635442 26131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:16:46.635563 26134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } }
I20260924 07:16:46.636222 26136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 743d90099c2543348ccde444ddb60ad4. Latest consensus state: current_term: 1 leader_uuid: "743d90099c2543348ccde444ddb60ad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } } }
I20260924 07:16:46.636229 26135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "743d90099c2543348ccde444ddb60ad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } } }
I20260924 07:16:46.636303 26136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:46.636333 26135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: This master's current role is: LEADER
I20260924 07:16:46.636651 26143 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:16:46.636714 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 26063
I20260924 07:16:46.636825 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:16:46.637037 10763 external_mini_cluster.cc:1428] Setting key b441543ec5170e4c58265fc864f768ee
I20260924 07:16:46.637190 26143 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:16:46.638734 26143 catalog_manager.cc:1347] Generated new cluster ID: 0e7969ba59b04c92b1097a130df4be4b
I20260924 07:16:46.638782 26143 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:16:46 dist-test-slave-7q7m krb5kdc[26050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:46.647737 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:46.637468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46066 (local address 127.10.130.254:39723)
0924 07:16:46.637758 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:46.637767 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:46.637798 (+ 31us) server_negotiation.cc:407] Connection header received
0924 07:16:46.637888 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:46.637893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:46.637948 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:46.638056 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:46.638753 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:46.640147 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:46.641163 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:46.641762 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:46.641840 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:46.644505 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:46.644509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:46.644510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:46.644520 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:46.646281 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:46.646650 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:46.646653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:46.646653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:46.646742 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:46.646918 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:46.646918 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:46.646919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:46.647186 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:46.647297 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:46.647411 (+ 114us) server_negotiation.cc:299] Negotiation successful
0924 07:16:46.647530 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":73,"threads_started":1}
I20260924 07:16:46.659427 26143 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:16:46.659921 26143 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:16:46.667236 26143 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Generated new TSK 0
I20260924 07:16:46.667384 26143 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:16:46.691226 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39723
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:46.768131 26157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:46.768319 26157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:46.768342 26157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:46.769449 26157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:46.769487 26157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:46.769516 26157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:16:46.770885 26157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:46.771109 26157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:46.771304 26157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:46.775774 26163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:46.775780 26165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:46.775925 26157 server_base.cc:1034] running on GCE node
W20260924 07:16:46.776007 26162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:46.776180 26157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:46.776384 26157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:46.777520 26157 hybrid_clock.cc:648] HybridClock initialized: now 1790234206777512 us; error 28 us; skew 500 ppm
Sep 24 07:16:46 dist-test-slave-7q7m krb5kdc[26050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234206, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:46.779772 26157 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:16:46.780195 26157 webserver.cc:460] Webserver started at http://127.10.130.193:41875/ using document root <none> and password file <none>
I20260924 07:16:46.780380 26157 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:46.780432 26157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:46.780548 26157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:16:46.781513 26157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "a06b3282a1ac4806950fbc48b9c0c1c8"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "2587d2694dc7bd991518eba1ff310f21"
server_key_iv: "a7f522e1b98c16198275e50833e81851"
server_key_version: "encryptionkey@0"
I20260924 07:16:46.781823 26157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "a06b3282a1ac4806950fbc48b9c0c1c8"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "2587d2694dc7bd991518eba1ff310f21"
server_key_iv: "a7f522e1b98c16198275e50833e81851"
server_key_version: "encryptionkey@0"
I20260924 07:16:46.782974 26157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:16:46.783552 26172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:46.783699 26157 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:16:46.783773 26157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "a06b3282a1ac4806950fbc48b9c0c1c8"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "2587d2694dc7bd991518eba1ff310f21"
server_key_iv: "a7f522e1b98c16198275e50833e81851"
server_key_version: "encryptionkey@0"
I20260924 07:16:46.783833 26157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:46.809392 26157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:46.811105 26157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:46.811237 26157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:46.811451 26157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:46.811738 26157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:46.811769 26157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:46.811797 26157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:46.811829 26157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:46.817260 26157 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:38027
I20260924 07:16:46.817328 26285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:38027 every 8 connection(s)
I20260924 07:16:46.817605 26157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
Sep 24 07:16:46 dist-test-slave-7q7m krb5kdc[26050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234206, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:46.825613 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 26157
I20260924 07:16:46.825730 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:16:46.825848 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:46.818259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:45325 (local address 127.10.130.254:39723)
0924 07:16:46.818317 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:46.818319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:46.818530 (+ 211us) server_negotiation.cc:407] Connection header received
0924 07:16:46.818981 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:46.818982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:46.818995 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:46.819050 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:46.819932 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:46.820476 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:46.821405 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:46.821611 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:46.821663 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:46.823538 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:46.823539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:46.823540 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:46.823548 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:46.824738 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:46.825079 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:46.825080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:46.825080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:46.825125 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:46.825295 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:46.825295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:46.825296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:46.825384 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:46.825432 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:46.825679 (+ 247us) server_negotiation.cc:299] Negotiation successful
0924 07:16:46.825763 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:16:46.825946 10763 external_mini_cluster.cc:1428] Setting key 0fadf84367ed97b33f32c18bd51b250b
I20260924 07:16:46.825966 26288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:46.818239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:45325)
0924 07:16:46.818394 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:46.818436 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:46.818902 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:46.819110 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:46.819113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:46.819660 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:46.819850 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:46.819855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:46.820547 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:46.820548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:46.821329 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:46.821331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:46.821672 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:46.821673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:46.821769 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:46.822183 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:46.822191 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:46.823456 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:46.824806 (+ 1350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:46.824812 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:46.824814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:46.825025 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:46.825167 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:46.825168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:46.825168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:46.825255 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:46.825430 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:46.825432 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:46.825542 (+ 110us) client_negotiation.cc:769] Sending connection context
0924 07:16:46.825619 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 07:16:46.825757 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260924 07:16:46.826479 26286 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39723
I20260924 07:16:46.826594 26286 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:46.826756 26286 heartbeater.cc:511] Master 127.10.130.254:39723 requested a full tablet report, sending...
I20260924 07:16:46.827258 26095 ts_manager.cc:194] Registered new tserver with Master: a06b3282a1ac4806950fbc48b9c0c1c8 (127.10.130.193:38027)
I20260924 07:16:46.828051 26095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:45325
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:16:46.847885 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39723
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:46.925522 26291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:46.925705 26291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:46.925726 26291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:46.926797 26291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:46.926831 26291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:46.926855 26291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:16:46.928220 26291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:46.928396 26291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:46.928574 26291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:46.933002 26296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:46.933130 26291 server_base.cc:1034] running on GCE node
W20260924 07:16:46.932989 26299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:46.932981 26297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:46.933423 26291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:46.933646 26291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:46.934777 26291 hybrid_clock.cc:648] HybridClock initialized: now 1790234206934762 us; error 26 us; skew 500 ppm
Sep 24 07:16:46 dist-test-slave-7q7m krb5kdc[26050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234206, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:46.937150 26291 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:16:46.937506 26291 webserver.cc:460] Webserver started at http://127.10.130.194:38725/ using document root <none> and password file <none>
I20260924 07:16:46.937677 26291 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:46.937714 26291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:46.937798 26291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:16:46.938706 26291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "8f8ae26193364c129e8ea00d8be67f84"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "622d3b1bcb051fa5273459bf6338846c"
server_key_iv: "6a3b85c32d27853336b581f0c301afc6"
server_key_version: "encryptionkey@0"
I20260924 07:16:46.938984 26291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "8f8ae26193364c129e8ea00d8be67f84"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "622d3b1bcb051fa5273459bf6338846c"
server_key_iv: "6a3b85c32d27853336b581f0c301afc6"
server_key_version: "encryptionkey@0"
I20260924 07:16:46.940089 26291 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:16:46.940704 26306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:46.940846 26291 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:16:46.940924 26291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "8f8ae26193364c129e8ea00d8be67f84"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "622d3b1bcb051fa5273459bf6338846c"
server_key_iv: "6a3b85c32d27853336b581f0c301afc6"
server_key_version: "encryptionkey@0"
I20260924 07:16:46.940979 26291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:46.951016 26291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:46.952940 26291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:46.953066 26291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:46.953229 26291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:46.953476 26291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:46.953514 26291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:46.953538 26291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:46.953554 26291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:46.958637 26291 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:36525
I20260924 07:16:46.958704 26419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:36525 every 8 connection(s)
I20260924 07:16:46.958954 26291 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:16:46.961944 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 26291
I20260924 07:16:46.962042 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:16:46.962229 10763 external_mini_cluster.cc:1428] Setting key 48071131e12f358f0d1e73954912ae46
Sep 24 07:16:46 dist-test-slave-7q7m krb5kdc[26050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234206, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:46.967655 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:46.959620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:57479 (local address 127.10.130.254:39723)
0924 07:16:46.959667 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:46.959669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:46.959834 (+ 165us) server_negotiation.cc:407] Connection header received
0924 07:16:46.960337 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:46.960338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:46.960347 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:46.960395 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:46.961229 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:46.961724 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:46.962922 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:46.963185 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:46.963248 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:46.965144 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:46.965146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:46.965146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:46.965153 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:46.966494 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:46.966821 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:46.966822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:46.966822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:46.966882 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:46.967074 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:46.967075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:46.967075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:46.967208 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:46.967276 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:46.967473 (+ 197us) server_negotiation.cc:299] Negotiation successful
0924 07:16:46.967567 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:16:46.967684 26422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:46.959579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:57479)
0924 07:16:46.959727 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:46.959766 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:46.960258 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:46.960459 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:46.960462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:46.960985 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:46.961167 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:46.961172 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:46.961797 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:46.961798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:46.962841 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:46.962845 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:46.963250 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:46.963251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:46.963348 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:46.963773 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:46.963781 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:46.965079 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:46.966562 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:46.966568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:46.966570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:46.966757 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:46.966939 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:46.966939 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:46.966939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:46.967022 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:46.967263 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:46.967265 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:46.967365 (+ 100us) client_negotiation.cc:769] Sending connection context
0924 07:16:46.967435 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 07:16:46.967538 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260924 07:16:46.968334 26420 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39723
I20260924 07:16:46.968418 26420 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:46.968571 26420 heartbeater.cc:511] Master 127.10.130.254:39723 requested a full tablet report, sending...
I20260924 07:16:46.968904 26095 ts_manager.cc:194] Registered new tserver with Master: 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525)
I20260924 07:16:46.969244 26095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:57479
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:16:46.984319 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39723
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:16:47.061507 26425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:16:47.061686 26425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:16:47.061707 26425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:16:47.062765 26425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:16:47.062799 26425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:16:47.062822 26425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:16:47.064169 26425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:16:47.064339 26425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:16:47.064505 26425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:16:47.068894 26431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:16:47.068957 26433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:47.068935 26425 server_base.cc:1034] running on GCE node
W20260924 07:16:47.068943 26430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:16:47.069284 26425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:16:47.069511 26425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:16:47.070644 26425 hybrid_clock.cc:648] HybridClock initialized: now 1790234207070632 us; error 25 us; skew 500 ppm
Sep 24 07:16:47 dist-test-slave-7q7m krb5kdc[26050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234207, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:16:47.072877 26425 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:16:47.073261 26425 webserver.cc:460] Webserver started at http://127.10.130.195:43945/ using document root <none> and password file <none>
I20260924 07:16:47.073446 26425 fs_manager.cc:359] Metadata directory not provided
I20260924 07:16:47.073486 26425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:16:47.073591 26425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:16:47.074506 26425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "f02e15281e8c458a98fc01a0c81e0945"
format_stamp: "Formatted at 2026-09-24 07:16:47 on dist-test-slave-7q7m"
server_key: "221e03abf06307dbdcddfeb20de4aada"
server_key_iv: "0be994f86dcc4c2f6dee6eba3f8a2f6e"
server_key_version: "encryptionkey@0"
I20260924 07:16:47.074800 26425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "f02e15281e8c458a98fc01a0c81e0945"
format_stamp: "Formatted at 2026-09-24 07:16:47 on dist-test-slave-7q7m"
server_key: "221e03abf06307dbdcddfeb20de4aada"
server_key_iv: "0be994f86dcc4c2f6dee6eba3f8a2f6e"
server_key_version: "encryptionkey@0"
I20260924 07:16:47.075899 26425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:16:47.076539 26440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:47.076674 26425 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:47.076743 26425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "f02e15281e8c458a98fc01a0c81e0945"
format_stamp: "Formatted at 2026-09-24 07:16:47 on dist-test-slave-7q7m"
server_key: "221e03abf06307dbdcddfeb20de4aada"
server_key_iv: "0be994f86dcc4c2f6dee6eba3f8a2f6e"
server_key_version: "encryptionkey@0"
I20260924 07:16:47.076803 26425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:16:47.092049 26425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:16:47.093540 26425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:16:47.093652 26425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:16:47.093828 26425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:16:47.094079 26425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:16:47.094110 26425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:47.094129 26425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:16:47.094152 26425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:16:47.099200 26425 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:39855
I20260924 07:16:47.099236 26553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:39855 every 8 connection(s)
I20260924 07:16:47.099493 26425 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
Sep 24 07:16:47 dist-test-slave-7q7m krb5kdc[26050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234207, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:16:47.107121 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.100152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:58419 (local address 127.10.130.254:39723)
0924 07:16:47.100208 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:47.100211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:47.100372 (+ 161us) server_negotiation.cc:407] Connection header received
0924 07:16:47.100799 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:47.100800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:47.100810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:47.100857 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:47.101656 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.102155 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.103029 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.103200 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.103239 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.105070 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:47.105072 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:47.105072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:47.105079 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:47.106130 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:47.106441 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:47.106441 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:47.106442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:47.106488 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:47.106659 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:47.106659 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:47.106659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:47.106750 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:47.106793 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:47.106998 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 07:16:47.107058 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 07:16:47.107304 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.100122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:58419)
0924 07:16:47.100274 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:47.100305 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:47.100727 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:47.100914 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:47.100917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:47.101420 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:16:47.101594 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:47.101598 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.102213 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.102213 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:47.102974 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:47.102976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.103248 (+ 272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.103249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:47.103340 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.103699 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:16:47.103706 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:16:47.104993 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:16:47.106189 (+ 1196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:47.106194 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:47.106196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:47.106389 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:47.106531 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:16:47.106531 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:16:47.106532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:16:47.106618 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:16:47.106791 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:16:47.106793 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:16:47.106900 (+ 107us) client_negotiation.cc:769] Sending connection context
0924 07:16:47.106973 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 07:16:47.107079 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260924 07:16:47.107740 26554 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39723
I20260924 07:16:47.107815 26554 heartbeater.cc:464] Registering TS with master...
I20260924 07:16:47.107949 26554 heartbeater.cc:511] Master 127.10.130.254:39723 requested a full tablet report, sending...
I20260924 07:16:47.108315 26095 ts_manager.cc:194] Registered new tserver with Master: f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195:39855)
I20260924 07:16:47.108397 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 26425
I20260924 07:16:47.108508 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:16:47.108683 10763 external_mini_cluster.cc:1428] Setting key 08342981da492df1f6f7d49827ce80f0
I20260924 07:16:47.108740 26095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:58419
I20260924 07:16:47.109421 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:16:47.115558 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.110639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46074 (local address 127.10.130.254:39723)
0924 07:16:47.110680 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:47.110682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:47.110790 (+ 108us) server_negotiation.cc:407] Connection header received
0924 07:16:47.110831 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:47.110832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:47.110841 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:47.110882 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:16:47.111354 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.111813 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.112532 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.112710 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.112752 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.113749 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:16:47.113750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:16:47.113750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:16:47.113756 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:16:47.114748 (+ 992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:47.115026 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:47.115027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:47.115027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:47.115071 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:16:47.115177 (+ 106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:16:47.115177 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:16:47.115178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:16:47.115302 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:16:47.115369 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:47.115439 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 07:16:47.115494 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:16:47.117033 26095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46074:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:16:47.118554 26095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:16:47.124333 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.120686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38526 (local address 127.10.130.195:39855)
0924 07:16:47.120874 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:47.120879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:47.120889 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:16:47.120939 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:47.120941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:47.120953 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:47.121016 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:47.121167 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.121967 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.123398 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.123926 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.124005 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.124064 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:47.124215 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 07:16:47.124256 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":67,"threads_started":1}
I20260924 07:16:47.124323 26562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.120489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:39855 (local address 127.0.0.1:38526)
0924 07:16:47.120834 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:47.120842 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:47.120877 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:47.121034 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:47.121037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:47.121039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:47.121106 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:47.121109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.122068 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.122069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:47.123307 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:47.123309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.124042 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.124043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:47.124122 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.124135 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 07:16:47.124194 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 07:16:47.124245 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":82,"threads_started":1}
I20260924 07:16:47.124639 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.120474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:36525 (local address 127.0.0.1:38320)
0924 07:16:47.120637 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:47.120674 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:47.120736 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:47.121299 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:47.121300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:47.121301 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:47.121333 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:47.121334 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.122570 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.122571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:47.123382 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:47.123384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.124363 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.124364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:47.124431 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.124437 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 07:16:47.124482 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 07:16:47.124523 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"spinlock_wait_cycles":896,"thread_start_us":59,"threads_started":1}
I20260924 07:16:47.124687 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.120586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38320 (local address 127.10.130.194:36525)
0924 07:16:47.121144 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:47.121148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:47.121157 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:16:47.121192 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:47.121194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:47.121208 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:47.121400 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:47.121474 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.122491 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.123457 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.124206 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.124275 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.124326 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:47.124568 (+ 242us) server_negotiation.cc:299] Negotiation successful
0924 07:16:47.124612 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":74,"threads_started":1}
I20260924 07:16:47.124712 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.120719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53942 (local address 127.10.130.193:38027)
0924 07:16:47.120931 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:47.120936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:47.120945 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:16:47.120981 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:47.120983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:47.120995 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:47.121070 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:47.121211 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.122334 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.123543 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.124219 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.124275 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.124326 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:47.124603 (+ 277us) server_negotiation.cc:299] Negotiation successful
0924 07:16:47.124643 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
I20260924 07:16:47.124805 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.120474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38027 (local address 127.0.0.1:53942)
0924 07:16:47.120744 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:47.120754 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:47.120814 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:47.121079 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:47.121080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:47.121082 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:47.121127 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:47.121129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.122431 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.122432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:47.123473 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:47.123475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.124361 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.124363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:47.124431 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.124438 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 07:16:47.124496 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 07:16:47.124524 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":7,"thread_start_us":61,"threads_started":1}
I20260924 07:16:47.125396 26354 tablet_service.cc:1467] Processing CreateTablet for tablet 8aa1343aa54f4d7da78a957703c7dcaa (DEFAULT_TABLE table=test-table [id=76b0e42bab424d57a6030704688945f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:16:47.125507 26220 tablet_service.cc:1467] Processing CreateTablet for tablet 8aa1343aa54f4d7da78a957703c7dcaa (DEFAULT_TABLE table=test-table [id=76b0e42bab424d57a6030704688945f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:16:47.125521 26488 tablet_service.cc:1467] Processing CreateTablet for tablet 8aa1343aa54f4d7da78a957703c7dcaa (DEFAULT_TABLE table=test-table [id=76b0e42bab424d57a6030704688945f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:16:47.125765 26220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aa1343aa54f4d7da78a957703c7dcaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:47.125759 26354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aa1343aa54f4d7da78a957703c7dcaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:47.125828 26488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8aa1343aa54f4d7da78a957703c7dcaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:16:47.128324 26568 tablet_bootstrap.cc:492] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945: Bootstrap starting.
I20260924 07:16:47.128847 26569 tablet_bootstrap.cc:492] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Bootstrap starting.
I20260924 07:16:47.129070 26570 tablet_bootstrap.cc:492] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Bootstrap starting.
I20260924 07:16:47.129179 26568 tablet_bootstrap.cc:654] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:47.129714 26570 tablet_bootstrap.cc:654] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:47.129755 26568 log.cc:824] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:47.130322 26569 tablet_bootstrap.cc:654] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Neither blocks nor log segments found. Creating new log.
I20260924 07:16:47.130525 26568 tablet_bootstrap.cc:492] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945: No bootstrap required, opened a new log
I20260924 07:16:47.130599 26568 ts_tablet_manager.cc:1397] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:16:47.130863 26569 log.cc:824] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:47.131491 26569 tablet_bootstrap.cc:492] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: No bootstrap required, opened a new log
I20260924 07:16:47.131549 26569 ts_tablet_manager.cc:1397] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 07:16:47.131619 26570 log.cc:824] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Log is configured to *not* fsync() on all Append() calls
I20260924 07:16:47.132426 26570 tablet_bootstrap.cc:492] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: No bootstrap required, opened a new log
I20260924 07:16:47.132494 26570 ts_tablet_manager.cc:1397] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 07:16:47.132527 26568 raft_consensus.cc:348] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } }
I20260924 07:16:47.132670 26568 raft_consensus.cc:374] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:47.132691 26568 raft_consensus.cc:729] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f02e15281e8c458a98fc01a0c81e0945, State: Initialized, Role: FOLLOWER
I20260924 07:16:47.132781 26568 consensus_queue.cc:260] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } }
I20260924 07:16:47.133037 26554 heartbeater.cc:503] Master 127.10.130.254:39723 was elected leader, sending a full tablet report...
I20260924 07:16:47.133219 26568 ts_tablet_manager.cc:1428] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:16:47.133571 26569 raft_consensus.cc:348] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } }
I20260924 07:16:47.133738 26569 raft_consensus.cc:374] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:47.133769 26569 raft_consensus.cc:729] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06b3282a1ac4806950fbc48b9c0c1c8, State: Initialized, Role: FOLLOWER
I20260924 07:16:47.133872 26569 consensus_queue.cc:260] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } }
I20260924 07:16:47.134140 26569 ts_tablet_manager.cc:1428] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260924 07:16:47.134203 26286 heartbeater.cc:503] Master 127.10.130.254:39723 was elected leader, sending a full tablet report...
I20260924 07:16:47.134323 26570 raft_consensus.cc:348] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } }
I20260924 07:16:47.134449 26570 raft_consensus.cc:374] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:16:47.134478 26570 raft_consensus.cc:729] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f8ae26193364c129e8ea00d8be67f84, State: Initialized, Role: FOLLOWER
I20260924 07:16:47.134574 26570 consensus_queue.cc:260] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } }
I20260924 07:16:47.134781 26570 ts_tablet_manager.cc:1428] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:16:47.134838 26420 heartbeater.cc:503] Master 127.10.130.254:39723 was elected leader, sending a full tablet report...
W20260924 07:16:47.209491 26421 tablet.cc:2369] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:16:47.231217 26574 raft_consensus.cc:482] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:16:47.231348 26574 raft_consensus.cc:504] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } }
I20260924 07:16:47.231698 26574 leader_election.cc:288] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525), a06b3282a1ac4806950fbc48b9c0c1c8 (127.10.130.193:38027)
I20260924 07:16:47.234346 26565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.232026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:40267 (local address 127.10.130.194:36525)
0924 07:16:47.232071 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:47.232073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:47.232080 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:16:47.232104 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:47.232105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:47.232114 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:47.232155 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:47.232293 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.232763 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.233667 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.234051 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.234111 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.234119 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:47.234248 (+ 129us) server_negotiation.cc:299] Negotiation successful
0924 07:16:47.234285 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 07:16:47.234346 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.231856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:36525 (local address 127.10.130.195:40267)
0924 07:16:47.231907 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:47.231920 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:47.232000 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:47.232169 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:47.232170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:47.232171 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:47.232238 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:47.232240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.232829 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.232830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:47.233586 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:47.233588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.234115 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.234116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:47.234191 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.234194 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:16:47.234238 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 07:16:47.234277 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":31}
I20260924 07:16:47.234414 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.232031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:59701 (local address 127.10.130.193:38027)
0924 07:16:47.232077 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:47.232079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:47.232124 (+ 45us) server_negotiation.cc:407] Connection header received
0924 07:16:47.232188 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:47.232189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:47.232198 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:47.232243 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:16:47.232386 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.232861 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.233747 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.234116 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.234173 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.234182 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:47.234314 (+ 132us) server_negotiation.cc:299] Negotiation successful
0924 07:16:47.234347 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:16:47.234458 26577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.231865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38027 (local address 127.10.130.195:59701)
0924 07:16:47.232074 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:47.232083 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:16:47.232139 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:16:47.232259 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:16:47.232261 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:16:47.232262 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:16:47.232332 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:47.232333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.232942 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.232943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:47.233672 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:16:47.233674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.234189 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.234190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:16:47.234259 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.234262 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:16:47.234304 (+ 42us) client_negotiation.cc:240] Negotiation successful
0924 07:16:47.234340 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":12,"thread_start_us":89,"threads_started":1}
I20260924 07:16:47.234714 26374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aa1343aa54f4d7da78a957703c7dcaa" candidate_uuid: "f02e15281e8c458a98fc01a0c81e0945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f8ae26193364c129e8ea00d8be67f84" is_pre_election: true
I20260924 07:16:47.234723 26240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aa1343aa54f4d7da78a957703c7dcaa" candidate_uuid: "f02e15281e8c458a98fc01a0c81e0945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" is_pre_election: true
I20260924 07:16:47.234855 26240 raft_consensus.cc:2454] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f02e15281e8c458a98fc01a0c81e0945 in term 0.
I20260924 07:16:47.234851 26374 raft_consensus.cc:2454] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f02e15281e8c458a98fc01a0c81e0945 in term 0.
I20260924 07:16:47.235071 26443 leader_election.cc:302] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f8ae26193364c129e8ea00d8be67f84, f02e15281e8c458a98fc01a0c81e0945; no voters:
I20260924 07:16:47.235217 26574 raft_consensus.cc:2781] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:16:47.235271 26574 raft_consensus.cc:482] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:16:47.235301 26574 raft_consensus.cc:3037] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:47.236021 26574 raft_consensus.cc:504] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } }
I20260924 07:16:47.236178 26574 leader_election.cc:288] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [CANDIDATE]: Term 1 election: Requested vote from peers 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525), a06b3282a1ac4806950fbc48b9c0c1c8 (127.10.130.193:38027)
I20260924 07:16:47.236322 26240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aa1343aa54f4d7da78a957703c7dcaa" candidate_uuid: "f02e15281e8c458a98fc01a0c81e0945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8"
I20260924 07:16:47.236343 26374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8aa1343aa54f4d7da78a957703c7dcaa" candidate_uuid: "f02e15281e8c458a98fc01a0c81e0945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f8ae26193364c129e8ea00d8be67f84"
I20260924 07:16:47.236397 26240 raft_consensus.cc:3037] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:47.236418 26374 raft_consensus.cc:3037] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:16:47.237156 26240 raft_consensus.cc:2454] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f02e15281e8c458a98fc01a0c81e0945 in term 1.
I20260924 07:16:47.237287 26374 raft_consensus.cc:2454] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f02e15281e8c458a98fc01a0c81e0945 in term 1.
I20260924 07:16:47.237303 26444 leader_election.cc:302] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a06b3282a1ac4806950fbc48b9c0c1c8, f02e15281e8c458a98fc01a0c81e0945; no voters:
I20260924 07:16:47.237372 26574 raft_consensus.cc:2781] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:16:47.237497 26574 raft_consensus.cc:686] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [term 1 LEADER]: Becoming Leader. State: Replica: f02e15281e8c458a98fc01a0c81e0945, State: Running, Role: LEADER
I20260924 07:16:47.237576 26574 consensus_queue.cc:237] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } }
I20260924 07:16:47.238286 26095 catalog_manager.cc:5495] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 reported cstate change: term changed from 0 to 1, leader changed from <none> to f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195). New cstate: current_term: 1 leader_uuid: "f02e15281e8c458a98fc01a0c81e0945" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } health_report { overall_health: HEALTHY } } }
I20260924 07:16:47.269778 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.267555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38542 (local address 127.10.130.195:39855)
0924 07:16:47.267612 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 07:16:47.267614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:16:47.267623 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:16:47.267663 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:16:47.267664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:16:47.267682 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:16:47.267743 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:16:47.267867 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.268335 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.269073 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:16:47.269254 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:16:47.269309 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:16:47.269461 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:16:47.269560 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:16:47.269630 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 07:16:47.269666 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 07:16:47.269686 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 07:16:47.271404 26374 raft_consensus.cc:1261] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84 [term 1 FOLLOWER]: Refusing update from remote peer f02e15281e8c458a98fc01a0c81e0945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:16:47.271456 26240 raft_consensus.cc:1261] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Refusing update from remote peer f02e15281e8c458a98fc01a0c81e0945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:16:47.271709 26578 consensus_queue.cc:1035] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:16:47.271831 26574 consensus_queue.cc:1035] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:16:47.274482 26583 mvcc.cc:205] Tried to move back new op lower bound from 7332799312981291008 to 7332799312845721600. Current Snapshot: MvccSnapshot[applied={T|T < 7332799312981291008 or (T in {7332799312981291008})}]
I20260924 07:16:47.275106 26582 mvcc.cc:205] Tried to move back new op lower bound from 7332799312981291008 to 7332799312845721600. Current Snapshot: MvccSnapshot[applied={T|T < 7332799312981291008}]
I20260924 07:16:47.276059 26094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:16:47.276140 26094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46074:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:16:47.276777 26094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Sending DeleteTablet for 3 replicas of tablet 8aa1343aa54f4d7da78a957703c7dcaa
I20260924 07:16:47.277081 26488 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aa1343aa54f4d7da78a957703c7dcaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:47 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:38526
I20260924 07:16:47.277078 26220 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aa1343aa54f4d7da78a957703c7dcaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:47 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:53942
I20260924 07:16:47.277272 26354 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aa1343aa54f4d7da78a957703c7dcaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:47 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:38320
I20260924 07:16:47.277298 26591 tablet_replica.cc:330] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945: stopping tablet replica
I20260924 07:16:47.277379 26591 raft_consensus.cc:2229] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:16:47.277498 26591 raft_consensus.cc:2258] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:16:47.277498 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 26157
I20260924 07:16:47.278693 26591 ts_tablet_manager.cc:1905] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:16:47.280388 26591 ts_tablet_manager.cc:1918] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:16:47.280457 26591 log.cc:1196] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/8aa1343aa54f4d7da78a957703c7dcaa
I20260924 07:16:47.280758 26591 ts_tablet_manager.cc:1939] T 8aa1343aa54f4d7da78a957703c7dcaa P f02e15281e8c458a98fc01a0c81e0945: Deleting consensus metadata
I20260924 07:16:47.281245 26080 catalog_manager.cc:4841] TS f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195:39855): tablet 8aa1343aa54f4d7da78a957703c7dcaa (table test-table [id=76b0e42bab424d57a6030704688945f0]) successfully deleted
W20260924 07:16:47.282691 26082 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:38027 (error 108)
I20260924 07:16:47.282950 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 26291
I20260924 07:16:47.283088 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.282989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38027 (local address 127.0.0.1:53948)
0924 07:16:47.283030 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:47.283060 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:38027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 07:16:47.284430 26082 catalog_manager.cc:4584] TS a06b3282a1ac4806950fbc48b9c0c1c8 (127.10.130.193:38027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8aa1343aa54f4d7da78a957703c7dcaa: Network error: Client connection negotiation failed: client connection to 127.10.130.193:38027: connect: Connection refused (error 111)
I20260924 07:16:47.287192 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 26425
I20260924 07:16:47.288751 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:16:47.288648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:36525 (local address 127.0.0.1:38328)
0924 07:16:47.288700 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 07:16:47.288725 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.194:36525: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 07:16:47.291157 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 26063
I20260924 07:17:07.295571 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39723
--webserver_interface=127.10.130.254
--webserver_port=36237
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:39723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:07.372702 26595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:07.372887 26595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:07.372915 26595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:07.374044 26595 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 07:17:07.374081 26595 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 07:17:07.374099 26595 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 07:17:07.374114 26595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:07.374128 26595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:07.374146 26595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:07.374168 26595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:07.374191 26595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:07.375586 26595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:39723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:07.375804 26595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:07.376013 26595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:07.383770 26602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:07.383813 26604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:07.383782 26601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:07.383818 26595 server_base.cc:1034] running on GCE node
I20260924 07:17:07.384260 26595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:07.384526 26595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:07.385669 26595 hybrid_clock.cc:648] HybridClock initialized: now 1790234227385654 us; error 30 us; skew 500 ppm
Sep 24 07:17:07 dist-test-slave-7q7m krb5kdc[26050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234227, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:07.392676 26595 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:17:07.393074 26595 webserver.cc:460] Webserver started at http://127.10.130.254:36237/ using document root <none> and password file <none>
I20260924 07:17:07.393257 26595 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:07.393313 26595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:07.394500 26595 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 07:17:07.395036 26611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:07.395179 26595 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:17:07.395255 26595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "743d90099c2543348ccde444ddb60ad4"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "9e6b7e14ef3d2466720c75e24edd42c4"
server_key_iv: "646e2e70cca402a27631d7a94712bf53"
server_key_version: "encryptionkey@0"
I20260924 07:17:07.395581 26595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:07.408221 26595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:07.409799 26595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:07.409922 26595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:07.413484 26595 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:39723
I20260924 07:17:07.413523 26663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:39723 every 8 connection(s)
I20260924 07:17:07.413833 26595 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:07.415115 26664 sys_catalog.cc:263] Verifying existing consensus state
I20260924 07:17:07.415699 26664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Bootstrap starting.
I20260924 07:17:07.419888 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 26595
I20260924 07:17:07.420231 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:38027
--local_ip_for_outbound_sockets=127.10.130.193
--tserver_master_addrs=127.10.130.254:39723
--webserver_port=41875
--webserver_interface=127.10.130.193
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:17:07.424103 26664 log.cc:824] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:07.425724 26664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 07:17:07.425967 26664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Bootstrap complete.
I20260924 07:17:07.427915 26664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } }
I20260924 07:17:07.428180 26664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 743d90099c2543348ccde444ddb60ad4, State: Initialized, Role: FOLLOWER
I20260924 07:17:07.428330 26664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } }
I20260924 07:17:07.428398 26664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:07.428431 26664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:07.428478 26664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 1 FOLLOWER]: Advancing to term 2
I20260924 07:17:07.429381 26664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } }
I20260924 07:17:07.429461 26664 leader_election.cc:302] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 743d90099c2543348ccde444ddb60ad4; no voters:
I20260924 07:17:07.429592 26664 leader_election.cc:288] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 07:17:07.429636 26668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 07:17:07.429785 26668 raft_consensus.cc:686] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 2 LEADER]: Becoming Leader. State: Replica: 743d90099c2543348ccde444ddb60ad4, State: Running, Role: LEADER
I20260924 07:17:07.429834 26664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:07.429867 26668 consensus_queue.cc:237] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } }
I20260924 07:17:07.430174 26670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 743d90099c2543348ccde444ddb60ad4. Latest consensus state: current_term: 2 leader_uuid: "743d90099c2543348ccde444ddb60ad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } } }
I20260924 07:17:07.430307 26670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:07.430485 26669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "743d90099c2543348ccde444ddb60ad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } } }
I20260924 07:17:07.430583 26669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:07.430595 26677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:07.431339 26677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:07.431437 26677 catalog_manager.cc:1259] Loaded cluster ID: 0e7969ba59b04c92b1097a130df4be4b
I20260924 07:17:07.431533 26677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:17:07.432560 26677 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:07.432690 26677 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Loaded TSK: 0
I20260924 07:17:07.449756 26677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Generated new TSK 1
I20260924 07:17:07.449839 26677 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 07:17:07.501257 26666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:07.501446 26666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:07.501477 26666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:07.502584 26666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:07.502621 26666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:07.502656 26666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:17:07.504045 26666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:38027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=41875
--tserver_master_addrs=127.10.130.254:39723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:07.504256 26666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:07.504444 26666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:07.509133 26691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:07.509141 26692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:07.509328 26666 server_base.cc:1034] running on GCE node
W20260924 07:17:07.512156 26694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:07.512348 26666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:07.512554 26666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:07.513684 26666 hybrid_clock.cc:648] HybridClock initialized: now 1790234227513672 us; error 30 us; skew 500 ppm
Sep 24 07:17:07 dist-test-slave-7q7m krb5kdc[26050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234227, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:07.515910 26666 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:17:07.516332 26666 webserver.cc:460] Webserver started at http://127.10.130.193:41875/ using document root <none> and password file <none>
I20260924 07:17:07.516513 26666 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:07.516573 26666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:07.517812 26666 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:17:07.518438 26701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:07.518584 26666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:07.518659 26666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "a06b3282a1ac4806950fbc48b9c0c1c8"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "2587d2694dc7bd991518eba1ff310f21"
server_key_iv: "a7f522e1b98c16198275e50833e81851"
server_key_version: "encryptionkey@0"
I20260924 07:17:07.518953 26666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:07.533725 26666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:07.535627 26666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:07.535744 26666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:07.535933 26666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:07.536353 26708 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 07:17:07.537640 26666 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 07:17:07.537688 26666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260924 07:17:07.537709 26666 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 07:17:07.538280 26666 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 07:17:07.538317 26666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 07:17:07.538376 26708 tablet_bootstrap.cc:492] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Bootstrap starting.
I20260924 07:17:07.545470 26666 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:38027
I20260924 07:17:07.545959 26666 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:17:07.548161 26815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:38027 every 8 connection(s)
I20260924 07:17:07.549214 26708 log.cc:824] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:07.551530 26708 tablet_bootstrap.cc:492] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 07:17:07.551759 26708 tablet_bootstrap.cc:492] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Bootstrap complete.
I20260924 07:17:07.552517 26708 ts_tablet_manager.cc:1397] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.006s
I20260924 07:17:07.553714 26708 raft_consensus.cc:348] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } }
I20260924 07:17:07.554006 26708 raft_consensus.cc:729] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06b3282a1ac4806950fbc48b9c0c1c8, State: Initialized, Role: FOLLOWER
I20260924 07:17:07.554160 26708 consensus_queue.cc:260] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } }
I20260924 07:17:07.554414 26708 ts_tablet_manager.cc:1428] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
Sep 24 07:17:07 dist-test-slave-7q7m krb5kdc[26050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234227, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:07.555675 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 26666
I20260924 07:17:07.555996 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:36525
--local_ip_for_outbound_sockets=127.10.130.194
--tserver_master_addrs=127.10.130.254:39723
--webserver_port=38725
--webserver_interface=127.10.130.194
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:17:07.559173 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.546743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:35911 (local address 127.10.130.254:39723)
0924 07:17:07.548470 (+ 1727us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:07.548480 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:07.548527 (+ 47us) server_negotiation.cc:407] Connection header received
0924 07:17:07.548684 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:07.548691 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:07.548755 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:07.548920 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:07.549676 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.551093 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.552582 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.553172 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.553282 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.555680 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:07.555685 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:07.555685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:07.555697 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:07.557534 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:07.557898 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:07.557901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:07.557902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:07.557993 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:07.558205 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:07.558206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:07.558206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:07.558436 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:07.558517 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:07.558861 (+ 344us) server_negotiation.cc:299] Negotiation successful
0924 07:17:07.558991 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1676,"thread_start_us":69,"threads_started":1}
I20260924 07:17:07.559173 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.546627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:35911)
0924 07:17:07.546796 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:07.546842 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:07.547411 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:07.548968 (+ 1557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:07.548973 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:07.549320 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:07.549567 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.549574 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.551226 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.551227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.552493 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.552496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.553278 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.553279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.553418 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.554019 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:07.554035 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:07.555576 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:07.557618 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:07.557624 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:07.557626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:07.557822 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:07.558052 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:07.558053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:07.558053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:07.558149 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:07.558517 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:07.558525 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:07.558682 (+ 157us) client_negotiation.cc:769] Sending connection context
0924 07:17:07.558798 (+ 116us) client_negotiation.cc:240] Negotiation successful
0924 07:17:07.558964 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260924 07:17:07.559793 26816 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39723
I20260924 07:17:07.559883 26816 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:07.560079 26816 heartbeater.cc:511] Master 127.10.130.254:39723 requested a full tablet report, sending...
I20260924 07:17:07.560739 26628 ts_manager.cc:194] Registered new tserver with Master: a06b3282a1ac4806950fbc48b9c0c1c8 (127.10.130.193:38027)
I20260924 07:17:07.561869 26628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:35911
I20260924 07:17:07.562988 26816 heartbeater.cc:503] Master 127.10.130.254:39723 was elected leader, sending a full tablet report...
I20260924 07:17:07.567000 26824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.562453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42984 (local address 127.10.130.193:38027)
0924 07:17:07.562600 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:07.562604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:07.563358 (+ 754us) server_negotiation.cc:407] Connection header received
0924 07:17:07.563394 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:07.563396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:07.563410 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:07.563466 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:07.563641 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.564682 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.565927 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.566633 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.566705 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.566752 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:07.566898 (+ 146us) server_negotiation.cc:299] Negotiation successful
0924 07:17:07.566934 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260924 07:17:07.567011 26823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.562245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38027 (local address 127.0.0.1:42984)
0924 07:17:07.563199 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:07.563234 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:07.563292 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:07.563516 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:07.563519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:07.563521 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:07.563574 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.563575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.564760 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.564761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.565838 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.565841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.566714 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.566715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.566799 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.566812 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 07:17:07.566882 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 07:17:07.566929 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"thread_start_us":73,"threads_started":1}
I20260924 07:17:07.567556 26750 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aa1343aa54f4d7da78a957703c7dcaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:47 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:42984
I20260924 07:17:07.568122 26825 tablet_replica.cc:330] stopping tablet replica
I20260924 07:17:07.568215 26825 raft_consensus.cc:2229] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:07.568279 26825 raft_consensus.cc:2258] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:07.568564 26825 ts_tablet_manager.cc:1905] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:07.570513 26825 ts_tablet_manager.cc:1918] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:17:07.570571 26825 log.cc:1196] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal/wals/8aa1343aa54f4d7da78a957703c7dcaa
I20260924 07:17:07.570755 26825 ts_tablet_manager.cc:1939] T 8aa1343aa54f4d7da78a957703c7dcaa P a06b3282a1ac4806950fbc48b9c0c1c8: Deleting consensus metadata
I20260924 07:17:07.571194 26615 catalog_manager.cc:4841] TS a06b3282a1ac4806950fbc48b9c0c1c8 (127.10.130.193:38027): tablet 8aa1343aa54f4d7da78a957703c7dcaa (table test-table [id=76b0e42bab424d57a6030704688945f0]) successfully deleted
W20260924 07:17:07.635291 26822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:07.635466 26822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:07.635488 26822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:07.636588 26822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:07.636623 26822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:07.636648 26822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:17:07.637971 26822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:36525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=38725
--tserver_master_addrs=127.10.130.254:39723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:07.638124 26822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:07.638281 26822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:07.642638 26833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:07.642616 26830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:07.642657 26822 server_base.cc:1034] running on GCE node
W20260924 07:17:07.642625 26831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:07.643008 26822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:07.643188 26822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:07.644307 26822 hybrid_clock.cc:648] HybridClock initialized: now 1790234227644298 us; error 30 us; skew 500 ppm
Sep 24 07:17:07 dist-test-slave-7q7m krb5kdc[26050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234227, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:07.646463 26822 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:17:07.646847 26822 webserver.cc:460] Webserver started at http://127.10.130.194:38725/ using document root <none> and password file <none>
I20260924 07:17:07.647019 26822 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:07.647075 26822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:07.648211 26822 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 07:17:07.648736 26840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:07.648872 26822 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:17:07.648944 26822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "8f8ae26193364c129e8ea00d8be67f84"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "622d3b1bcb051fa5273459bf6338846c"
server_key_iv: "6a3b85c32d27853336b581f0c301afc6"
server_key_version: "encryptionkey@0"
I20260924 07:17:07.649206 26822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:07.681226 26822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:07.682858 26822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:07.682988 26822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:07.683189 26822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:07.683574 26847 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 07:17:07.684960 26822 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 07:17:07.684999 26822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 07:17:07.685030 26822 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 07:17:07.685611 26822 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 07:17:07.685639 26822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 07:17:07.685685 26847 tablet_bootstrap.cc:492] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Bootstrap starting.
I20260924 07:17:07.691917 26822 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:36525
I20260924 07:17:07.692029 26954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:36525 every 8 connection(s)
I20260924 07:17:07.692307 26822 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:17:07.694152 26847 log.cc:824] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:07.696301 26847 tablet_bootstrap.cc:492] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 07:17:07.696573 26847 tablet_bootstrap.cc:492] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Bootstrap complete.
I20260924 07:17:07.697427 26847 ts_tablet_manager.cc:1397] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
Sep 24 07:17:07 dist-test-slave-7q7m krb5kdc[26050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234227, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:07.698498 26847 raft_consensus.cc:348] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } }
I20260924 07:17:07.698733 26847 raft_consensus.cc:729] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f8ae26193364c129e8ea00d8be67f84, State: Initialized, Role: FOLLOWER
I20260924 07:17:07.698870 26847 consensus_queue.cc:260] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } } peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } }
I20260924 07:17:07.699129 26847 ts_tablet_manager.cc:1428] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260924 07:17:07.701233 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.693004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:38095 (local address 127.10.130.254:39723)
0924 07:17:07.693069 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:07.693071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:07.693267 (+ 196us) server_negotiation.cc:407] Connection header received
0924 07:17:07.693944 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:07.693945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:07.693956 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:07.693994 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:07.694680 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.695175 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.696597 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.696810 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.696856 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.699152 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:07.699154 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:07.699154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:07.699162 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:07.700257 (+ 1095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:07.700583 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:07.700584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:07.700584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:07.700629 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:07.700782 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:07.700782 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:07.700782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:07.700869 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:07.700910 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:07.701108 (+ 198us) server_negotiation.cc:299] Negotiation successful
0924 07:17:07.701170 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 07:17:07.701354 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 26822
I20260924 07:17:07.701418 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.692929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:38095)
0924 07:17:07.693110 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:07.693171 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:07.693823 (+ 652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:07.694066 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:07.694070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:07.694384 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:07.694598 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.694604 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.695277 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.695279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.696518 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.696520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.696892 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.696893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.697023 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.697472 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:07.697481 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:07.699022 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:07.700319 (+ 1297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:07.700325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:07.700326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:07.700527 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:07.700663 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:07.700664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:07.700664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:07.700743 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:07.700923 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:07.700926 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:07.701024 (+ 98us) client_negotiation.cc:769] Sending connection context
0924 07:17:07.701096 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 07:17:07.701234 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260924 07:17:07.701634 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:39855
--local_ip_for_outbound_sockets=127.10.130.195
--tserver_master_addrs=127.10.130.254:39723
--webserver_port=43945
--webserver_interface=127.10.130.195
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:17:07.701921 26955 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39723
I20260924 07:17:07.702004 26955 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:07.702149 26955 heartbeater.cc:511] Master 127.10.130.254:39723 requested a full tablet report, sending...
I20260924 07:17:07.702647 26628 ts_manager.cc:194] Registered new tserver with Master: 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525)
I20260924 07:17:07.703146 26628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:38095
I20260924 07:17:07.703868 26955 heartbeater.cc:503] Master 127.10.130.254:39723 was elected leader, sending a full tablet report...
Sep 24 07:17:07 dist-test-slave-7q7m krb5kdc[26050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234227, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for kudu/127.10.130.194@KRBTEST.COM
I20260924 07:17:07.709877 26961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.703099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38324 (local address 127.10.130.194:36525)
0924 07:17:07.703234 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:07.703238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:07.703248 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:17:07.703278 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:07.703280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:07.703293 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:07.703360 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:07.703720 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.704496 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.705171 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.705454 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.705508 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.707387 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:07.707393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:07.707393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:07.707398 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:07.708744 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:07.709154 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:07.709156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:07.709156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:07.709199 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:07.709423 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:07.709424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:07.709424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:07.709537 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:07.709592 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:07.709759 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 07:17:07.709821 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260924 07:17:07.709939 26823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.702939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:36525 (local address 127.0.0.1:38324)
0924 07:17:07.702987 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:07.702996 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:07.703029 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:07.703362 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:07.703363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:07.703622 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:07.703651 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.703652 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.704563 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.704564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.705105 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.705107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.705518 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.705519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.705576 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.706066 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:07.706071 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:07.707310 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:07.708820 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:07.708829 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:07.708831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:07.709084 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:07.709296 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:07.709296 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:07.709297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:07.709362 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:07.709595 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:07.709603 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:07.709656 (+ 53us) client_negotiation.cc:769] Sending connection context
0924 07:17:07.709729 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 07:17:07.709824 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 07:17:07.710291 26889 tablet_service.cc:1514] Processing DeleteTablet for tablet 8aa1343aa54f4d7da78a957703c7dcaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:16:47 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:38324
I20260924 07:17:07.710798 26962 tablet_replica.cc:330] stopping tablet replica
I20260924 07:17:07.710875 26962 raft_consensus.cc:2229] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:07.710937 26962 raft_consensus.cc:2258] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:07.711179 26962 ts_tablet_manager.cc:1905] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:07.713070 26962 ts_tablet_manager.cc:1918] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:17:07.713125 26962 log.cc:1196] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/8aa1343aa54f4d7da78a957703c7dcaa
I20260924 07:17:07.713299 26962 ts_tablet_manager.cc:1939] T 8aa1343aa54f4d7da78a957703c7dcaa P 8f8ae26193364c129e8ea00d8be67f84: Deleting consensus metadata
I20260924 07:17:07.713678 26614 catalog_manager.cc:4841] TS 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525): tablet 8aa1343aa54f4d7da78a957703c7dcaa (table test-table [id=76b0e42bab424d57a6030704688945f0]) successfully deleted
W20260924 07:17:07.779693 26960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:07.779871 26960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:07.779893 26960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:07.781260 26960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:07.781327 26960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:07.781356 26960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:17:07.782743 26960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:39855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=43945
--tserver_master_addrs=127.10.130.254:39723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:07.782912 26960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:07.783079 26960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:07.787537 26967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:07.787652 26960 server_base.cc:1034] running on GCE node
W20260924 07:17:07.787554 26970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:07.787554 26968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:07.787946 26960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:07.788223 26960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:07.789348 26960 hybrid_clock.cc:648] HybridClock initialized: now 1790234227789339 us; error 27 us; skew 500 ppm
Sep 24 07:17:07 dist-test-slave-7q7m krb5kdc[26050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234227, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:07.791528 26960 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:17:07.791864 26960 webserver.cc:460] Webserver started at http://127.10.130.195:43945/ using document root <none> and password file <none>
I20260924 07:17:07.792064 26960 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:07.792119 26960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:07.793255 26960 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 07:17:07.793740 26977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:07.793872 26960 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:07.793949 26960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "f02e15281e8c458a98fc01a0c81e0945"
format_stamp: "Formatted at 2026-09-24 07:16:47 on dist-test-slave-7q7m"
server_key: "221e03abf06307dbdcddfeb20de4aada"
server_key_iv: "0be994f86dcc4c2f6dee6eba3f8a2f6e"
server_key_version: "encryptionkey@0"
I20260924 07:17:07.794238 26960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:07.808290 26960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:07.809935 26960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:07.810043 26960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:07.810210 26960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:07.810510 26960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:07.810544 26960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:07.810564 26960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:07.810580 26960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:07.815706 26960 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:39855
I20260924 07:17:07.815764 27090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:39855 every 8 connection(s)
I20260924 07:17:07.816107 26960 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
Sep 24 07:17:07 dist-test-slave-7q7m krb5kdc[26050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234227, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:07.823884 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.816696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:48569 (local address 127.10.130.254:39723)
0924 07:17:07.816751 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:07.816753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:07.816937 (+ 184us) server_negotiation.cc:407] Connection header received
0924 07:17:07.817412 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:07.817413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:07.817426 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:07.817475 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:07.818341 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.818820 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.819764 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.819964 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.820066 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.821908 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:07.821910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:07.821910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:07.821917 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:07.822929 (+ 1012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:07.823239 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:07.823240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:07.823240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:07.823282 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:07.823448 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:07.823449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:07.823449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:07.823538 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:07.823580 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:07.823768 (+ 188us) server_negotiation.cc:299] Negotiation successful
0924 07:17:07.823822 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 07:17:07.824052 27093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.816696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:48569)
0924 07:17:07.816827 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:07.816858 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:07.817318 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:07.817550 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:07.817554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:07.818070 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:07.818246 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.818250 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.818898 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.818899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.819680 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.819683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.820072 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.820073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.820193 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.820555 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:07.820562 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:07.821835 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:07.822990 (+ 1155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:07.822994 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:07.822996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:07.823186 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:07.823324 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:07.823325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:07.823325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:07.823410 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:07.823577 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:07.823580 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:07.823679 (+ 99us) client_negotiation.cc:769] Sending connection context
0924 07:17:07.823751 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 07:17:07.823854 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":66,"threads_started":1}
I20260924 07:17:07.824409 27091 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39723
I20260924 07:17:07.824491 27091 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:07.824640 27091 heartbeater.cc:511] Master 127.10.130.254:39723 requested a full tablet report, sending...
I20260924 07:17:07.824949 26628 ts_manager.cc:194] Registered new tserver with Master: f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195:39855)
I20260924 07:17:07.825258 26628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:48569
I20260924 07:17:07.826207 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 26960
I20260924 07:17:07.831472 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.826528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57348 (local address 127.10.130.254:39723)
0924 07:17:07.826559 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:07.826561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:07.826689 (+ 128us) server_negotiation.cc:407] Connection header received
0924 07:17:07.826738 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:07.826739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:07.826746 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:07.826789 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:07.827290 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.827719 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.828432 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.828602 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.828645 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.829662 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:07.829662 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:07.829663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:07.829667 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:07.830696 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:07.830982 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:07.830983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:07.830983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:07.831047 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:07.831164 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:07.831164 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:07.831164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:07.831243 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:07.831344 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:07.831369 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 07:17:07.831416 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:17:07.832144 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:17:07.834298 26819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.832413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57356 (local address 127.10.130.254:39723)
0924 07:17:07.832452 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:07.832454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:07.832552 (+ 98us) server_negotiation.cc:407] Connection header received
0924 07:17:07.832598 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:07.832599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:07.832625 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:07.832672 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:17:07.832748 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.833167 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.833816 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.833974 (+ 158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.834018 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.834108 (+ 90us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:17:07.834184 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:17:07.834232 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:07.834246 (+ 14us) server_negotiation.cc:299] Negotiation successful
0924 07:17:07.834257 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 07:17:07.834522 26628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:17:07.835958 26628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:17:07.837710 26889 tablet_service.cc:1467] Processing CreateTablet for tablet de450178831446498554a40bb9d09fe3 (DEFAULT_TABLE table=test-table [id=a3f434b271724681a9411d7554dead04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:07.837734 26750 tablet_service.cc:1467] Processing CreateTablet for tablet de450178831446498554a40bb9d09fe3 (DEFAULT_TABLE table=test-table [id=a3f434b271724681a9411d7554dead04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:07.837888 26750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de450178831446498554a40bb9d09fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:07.837888 26889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de450178831446498554a40bb9d09fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:07.839342 26847 tablet_bootstrap.cc:492] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84: Bootstrap starting.
I20260924 07:17:07.839638 26847 tablet_bootstrap.cc:654] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:07.839861 26708 tablet_bootstrap.cc:492] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8: Bootstrap starting.
I20260924 07:17:07.840174 26708 tablet_bootstrap.cc:654] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:07.840556 26847 tablet_bootstrap.cc:492] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84: No bootstrap required, opened a new log
I20260924 07:17:07.840588 26823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.837518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:39855 (local address 127.0.0.1:42082)
0924 07:17:07.837550 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:07.837559 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:07.837592 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:07.837946 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:07.837946 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:07.837951 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:07.837991 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.837992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.838830 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.838831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.839630 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.839634 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.840367 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.840368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.840439 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.840442 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:17:07.840494 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 07:17:07.840527 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260924 07:17:07.840600 26847 ts_tablet_manager.cc:1397] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 07:17:07.840605 27096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.837629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42082 (local address 127.10.130.195:39855)
0924 07:17:07.837778 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:07.837782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:07.837791 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:17:07.837828 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:07.837830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:07.837844 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:07.837940 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:07.838049 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.838761 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.839717 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.840294 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.840359 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.840411 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:07.840507 (+ 96us) server_negotiation.cc:299] Negotiation successful
0924 07:17:07.840539 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
I20260924 07:17:07.840737 26847 raft_consensus.cc:348] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:07.840811 26847 raft_consensus.cc:374] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:07.840854 26847 raft_consensus.cc:729] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f8ae26193364c129e8ea00d8be67f84, State: Initialized, Role: FOLLOWER
I20260924 07:17:07.840906 26847 consensus_queue.cc:260] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:07.840992 26847 ts_tablet_manager.cc:1428] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:07.841274 26708 tablet_bootstrap.cc:492] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8: No bootstrap required, opened a new log
I20260924 07:17:07.841320 26708 ts_tablet_manager.cc:1397] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 07:17:07.841300 27025 tablet_service.cc:1467] Processing CreateTablet for tablet de450178831446498554a40bb9d09fe3 (DEFAULT_TABLE table=test-table [id=a3f434b271724681a9411d7554dead04]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:07.841414 26708 raft_consensus.cc:348] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:07.841475 26708 raft_consensus.cc:374] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:07.841530 26708 raft_consensus.cc:729] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06b3282a1ac4806950fbc48b9c0c1c8, State: Initialized, Role: FOLLOWER
I20260924 07:17:07.841590 26708 consensus_queue.cc:260] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:07.841635 27025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de450178831446498554a40bb9d09fe3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:07.841676 26708 ts_tablet_manager.cc:1428] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:07.844121 27099 tablet_bootstrap.cc:492] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945: Bootstrap starting.
I20260924 07:17:07.844628 27099 tablet_bootstrap.cc:654] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:07.845017 27099 log.cc:824] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:07.845553 27099 tablet_bootstrap.cc:492] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945: No bootstrap required, opened a new log
I20260924 07:17:07.845609 27099 ts_tablet_manager.cc:1397] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:17:07.846977 27099 raft_consensus.cc:348] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:07.847118 27099 raft_consensus.cc:374] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:07.847157 27099 raft_consensus.cc:729] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f02e15281e8c458a98fc01a0c81e0945, State: Initialized, Role: FOLLOWER
I20260924 07:17:07.847237 27099 consensus_queue.cc:260] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:07.847421 27099 ts_tablet_manager.cc:1428] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:17:07.847471 27091 heartbeater.cc:503] Master 127.10.130.254:39723 was elected leader, sending a full tablet report...
I20260924 07:17:07.874482 27101 raft_consensus.cc:482] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:17:07.874572 27101 raft_consensus.cc:504] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:07.874881 27101 leader_election.cc:288] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525), a06b3282a1ac4806950fbc48b9c0c1c8 (127.10.130.193:38027)
I20260924 07:17:07.877493 26961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.875138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:59941 (local address 127.10.130.194:36525)
0924 07:17:07.875180 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:07.875181 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:07.875188 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:17:07.875210 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:07.875210 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:07.875226 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:07.875276 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:07.875413 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.875903 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.876779 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.877180 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.877243 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.877316 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:07.877392 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 07:17:07.877430 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 07:17:07.877525 27093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.874994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:36525 (local address 127.10.130.195:59941)
0924 07:17:07.875066 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:07.875077 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:07.875114 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:07.875290 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:07.875291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:07.875292 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:07.875351 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.875353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.876003 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.876004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.876689 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.876692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.877256 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.877257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.877331 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.877334 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:17:07.877381 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 07:17:07.877423 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 07:17:07.877519 26824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.875148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:54621 (local address 127.10.130.193:38027)
0924 07:17:07.875189 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:07.875191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:07.875280 (+ 89us) server_negotiation.cc:407] Connection header received
0924 07:17:07.875331 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:07.875332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:07.875342 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:07.875382 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:07.875513 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.875997 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.876858 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.877216 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.877274 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.877282 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:07.877423 (+ 141us) server_negotiation.cc:299] Negotiation successful
0924 07:17:07.877458 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:17:07.877543 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.875031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38027 (local address 127.10.130.195:54621)
0924 07:17:07.875217 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:07.875226 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:07.875281 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:07.875392 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:07.875393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:07.875394 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:07.875458 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.875460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.876058 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.876058 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.876777 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:07.876779 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.877284 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.877285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:07.877357 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.877360 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:17:07.877403 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 07:17:07.877443 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260924 07:17:07.877923 26909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de450178831446498554a40bb9d09fe3" candidate_uuid: "f02e15281e8c458a98fc01a0c81e0945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f8ae26193364c129e8ea00d8be67f84" is_pre_election: true
I20260924 07:17:07.877959 26770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de450178831446498554a40bb9d09fe3" candidate_uuid: "f02e15281e8c458a98fc01a0c81e0945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" is_pre_election: true
I20260924 07:17:07.878050 26909 raft_consensus.cc:2454] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f02e15281e8c458a98fc01a0c81e0945 in term 0.
I20260924 07:17:07.878093 26770 raft_consensus.cc:2454] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f02e15281e8c458a98fc01a0c81e0945 in term 0.
I20260924 07:17:07.878304 26980 leader_election.cc:302] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f8ae26193364c129e8ea00d8be67f84, f02e15281e8c458a98fc01a0c81e0945; no voters:
I20260924 07:17:07.878454 27101 raft_consensus.cc:2781] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:17:07.878492 27101 raft_consensus.cc:482] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:17:07.878515 27101 raft_consensus.cc:3037] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:07.879137 27101 raft_consensus.cc:504] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:07.879240 27101 leader_election.cc:288] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [CANDIDATE]: Term 1 election: Requested vote from peers 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525), a06b3282a1ac4806950fbc48b9c0c1c8 (127.10.130.193:38027)
I20260924 07:17:07.879371 26909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de450178831446498554a40bb9d09fe3" candidate_uuid: "f02e15281e8c458a98fc01a0c81e0945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f8ae26193364c129e8ea00d8be67f84"
I20260924 07:17:07.879402 26770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de450178831446498554a40bb9d09fe3" candidate_uuid: "f02e15281e8c458a98fc01a0c81e0945" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8"
I20260924 07:17:07.879447 26909 raft_consensus.cc:3037] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:07.879462 26770 raft_consensus.cc:3037] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:07.880347 26909 raft_consensus.cc:2454] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f02e15281e8c458a98fc01a0c81e0945 in term 1.
I20260924 07:17:07.880510 26980 leader_election.cc:302] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f8ae26193364c129e8ea00d8be67f84, f02e15281e8c458a98fc01a0c81e0945; no voters:
I20260924 07:17:07.880605 27101 raft_consensus.cc:2781] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:07.880620 26770 raft_consensus.cc:2454] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f02e15281e8c458a98fc01a0c81e0945 in term 1.
I20260924 07:17:07.880759 27101 raft_consensus.cc:686] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [term 1 LEADER]: Becoming Leader. State: Replica: f02e15281e8c458a98fc01a0c81e0945, State: Running, Role: LEADER
I20260924 07:17:07.880846 27101 consensus_queue.cc:237] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:07.881588 26627 catalog_manager.cc:5495] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 reported cstate change: term changed from 0 to 1, leader changed from <none> to f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195). New cstate: current_term: 1 leader_uuid: "f02e15281e8c458a98fc01a0c81e0945" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } health_report { overall_health: UNKNOWN } } }
I20260924 07:17:07.886276 27096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.884474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42084 (local address 127.10.130.195:39855)
0924 07:17:07.884524 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:07.884525 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:07.884533 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:17:07.884557 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:07.884558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:07.884571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:07.884624 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:17:07.884694 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.885120 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.885742 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:07.885905 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:07.885944 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:07.886042 (+ 98us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:17:07.886122 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:17:07.886168 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:07.886183 (+ 15us) server_negotiation.cc:299] Negotiation successful
0924 07:17:07.886197 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 07:17:07.887740 26909 raft_consensus.cc:1261] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84 [term 1 FOLLOWER]: Refusing update from remote peer f02e15281e8c458a98fc01a0c81e0945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:07.887770 26770 raft_consensus.cc:1261] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Refusing update from remote peer f02e15281e8c458a98fc01a0c81e0945: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:07.888074 27103 consensus_queue.cc:1035] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:07.888187 27101 consensus_queue.cc:1035] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:07.890671 27105 mvcc.cc:205] Tried to move back new op lower bound from 7332799397426008064 to 7332799397400621056. Current Snapshot: MvccSnapshot[applied={T|T < 7332799397426008064 or (T in {7332799397426008064})}]
I20260924 07:17:07.891206 27106 mvcc.cc:205] Tried to move back new op lower bound from 7332799397426008064 to 7332799397400621056. Current Snapshot: MvccSnapshot[applied={T|T < 7332799397426008064}]
I20260924 07:17:07.892254 26628 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:07.892330 26628 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:07.892958 26628 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Sending DeleteTablet for 3 replicas of tablet de450178831446498554a40bb9d09fe3
I20260924 07:17:07.893224 27025 tablet_service.cc:1514] Processing DeleteTablet for tablet de450178831446498554a40bb9d09fe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:07 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:42082
I20260924 07:17:07.893245 26889 tablet_service.cc:1514] Processing DeleteTablet for tablet de450178831446498554a40bb9d09fe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:07 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:38324
I20260924 07:17:07.893366 26750 tablet_service.cc:1514] Processing DeleteTablet for tablet de450178831446498554a40bb9d09fe3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:07 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:42984
I20260924 07:17:07.893378 26962 tablet_replica.cc:330] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84: stopping tablet replica
I20260924 07:17:07.893442 26962 raft_consensus.cc:2229] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:07.893453 26825 tablet_replica.cc:330] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8: stopping tablet replica
I20260924 07:17:07.893481 26962 raft_consensus.cc:2258] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:07.893492 26825 raft_consensus.cc:2229] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:07.893527 27114 tablet_replica.cc:330] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945: stopping tablet replica
I20260924 07:17:07.893543 26825 raft_consensus.cc:2258] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:07.893603 27114 raft_consensus.cc:2229] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:17:07.893684 27114 raft_consensus.cc:2258] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:07.893700 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 26666
I20260924 07:17:07.893846 26825 ts_tablet_manager.cc:1905] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:07.894531 27114 ts_tablet_manager.cc:1905] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:07.896493 27114 ts_tablet_manager.cc:1918] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:17:07.896513 26962 ts_tablet_manager.cc:1905] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:07.896562 27114 log.cc:1196] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/de450178831446498554a40bb9d09fe3
I20260924 07:17:07.896824 27114 ts_tablet_manager.cc:1939] T de450178831446498554a40bb9d09fe3 P f02e15281e8c458a98fc01a0c81e0945: Deleting consensus metadata
I20260924 07:17:07.897202 26613 catalog_manager.cc:4841] TS f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195:39855): tablet de450178831446498554a40bb9d09fe3 (table test-table [id=a3f434b271724681a9411d7554dead04]) successfully deleted
I20260924 07:17:07.897966 26962 ts_tablet_manager.cc:1918] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:17:07.898036 26962 log.cc:1196] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/de450178831446498554a40bb9d09fe3
I20260924 07:17:07.898227 26962 ts_tablet_manager.cc:1939] T de450178831446498554a40bb9d09fe3 P 8f8ae26193364c129e8ea00d8be67f84: Deleting consensus metadata
I20260924 07:17:07.898581 26614 catalog_manager.cc:4841] TS 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525): tablet de450178831446498554a40bb9d09fe3 (table test-table [id=a3f434b271724681a9411d7554dead04]) successfully deleted
W20260924 07:17:07.899940 26615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:38027 (error 108)
I20260924 07:17:07.900198 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 26822
I20260924 07:17:07.902235 26823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:07.902111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38027 (local address 127.0.0.1:40684)
0924 07:17:07.902163 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:07.902209 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:38027: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260924 07:17:07.902328 26615 catalog_manager.cc:4584] TS a06b3282a1ac4806950fbc48b9c0c1c8 (127.10.130.193:38027): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet de450178831446498554a40bb9d09fe3: Network error: Client connection negotiation failed: client connection to 127.10.130.193:38027: connect: Connection refused (error 111)
I20260924 07:17:07.905273 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 26960
I20260924 07:17:07.909391 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 26595
I20260924 07:17:27.914369 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39723
--webserver_interface=127.10.130.254
--webserver_port=36237
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:39723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:27.994210 27116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:27.994401 27116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:27.994424 27116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:27.995496 27116 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 07:17:27.995533 27116 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 07:17:27.995548 27116 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 07:17:27.995572 27116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:27.995594 27116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:27.995608 27116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:27.995621 27116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:27.995633 27116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:27.997035 27116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:39723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=36237
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:27.997224 27116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:27.997391 27116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:28.002122 27121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:28.002152 27116 server_base.cc:1034] running on GCE node
W20260924 07:17:28.002149 27122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:28.002202 27124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:28.002480 27116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:28.002810 27116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:28.003958 27116 hybrid_clock.cc:648] HybridClock initialized: now 1790234248003948 us; error 28 us; skew 500 ppm
Sep 24 07:17:28 dist-test-slave-7q7m krb5kdc[26050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234248, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:28.011059 27116 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:17:28.011505 27116 webserver.cc:460] Webserver started at http://127.10.130.254:36237/ using document root <none> and password file <none>
I20260924 07:17:28.011713 27116 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:28.011759 27116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:28.013103 27116 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 07:17:28.013739 27131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:28.013904 27116 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:28.013981 27116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "743d90099c2543348ccde444ddb60ad4"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "9e6b7e14ef3d2466720c75e24edd42c4"
server_key_iv: "646e2e70cca402a27631d7a94712bf53"
server_key_version: "encryptionkey@0"
I20260924 07:17:28.014312 27116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:28.025265 27116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:28.026993 27116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:28.027105 27116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:28.030701 27116 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:39723
I20260924 07:17:28.030735 27183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:39723 every 8 connection(s)
I20260924 07:17:28.031059 27116 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:28.032459 27184 sys_catalog.cc:263] Verifying existing consensus state
I20260924 07:17:28.033058 27184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Bootstrap starting.
I20260924 07:17:28.039016 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 27116
I20260924 07:17:28.039287 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:38027
--local_ip_for_outbound_sockets=127.10.130.193
--tserver_master_addrs=127.10.130.254:39723
--webserver_port=41875
--webserver_interface=127.10.130.193
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:17:28.040940 27184 log.cc:824] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:28.043068 27184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 07:17:28.043313 27184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Bootstrap complete.
I20260924 07:17:28.045323 27184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } }
I20260924 07:17:28.045532 27184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 743d90099c2543348ccde444ddb60ad4, State: Initialized, Role: FOLLOWER
I20260924 07:17:28.045686 27184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } }
I20260924 07:17:28.045758 27184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:28.045805 27184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:28.045854 27184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 2 FOLLOWER]: Advancing to term 3
I20260924 07:17:28.046767 27184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } }
I20260924 07:17:28.046854 27184 leader_election.cc:302] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 743d90099c2543348ccde444ddb60ad4; no voters:
I20260924 07:17:28.047014 27184 leader_election.cc:288] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 07:17:28.047060 27188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 07:17:28.047254 27188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [term 3 LEADER]: Becoming Leader. State: Replica: 743d90099c2543348ccde444ddb60ad4, State: Running, Role: LEADER
I20260924 07:17:28.047257 27184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:28.047405 27188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } }
I20260924 07:17:28.047700 27190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 743d90099c2543348ccde444ddb60ad4. Latest consensus state: current_term: 3 leader_uuid: "743d90099c2543348ccde444ddb60ad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } } }
I20260924 07:17:28.047708 27189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "743d90099c2543348ccde444ddb60ad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "743d90099c2543348ccde444ddb60ad4" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39723 } } }
I20260924 07:17:28.047791 27190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:28.047801 27189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:28.048031 27196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:28.048812 27196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:28.048902 27196 catalog_manager.cc:1259] Loaded cluster ID: 0e7969ba59b04c92b1097a130df4be4b
I20260924 07:17:28.048978 27196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:17:28.050418 27196 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:28.050585 27196 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Loaded TSK: 0
I20260924 07:17:28.050627 27196 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 743d90099c2543348ccde444ddb60ad4: Loaded TSK: 1
F20260924 07:17:28.050843 27196 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa343ca39ad google::LogMessage::Fail() at ??:0
@ 0x7fa343ca688a google::LogMessage::SendToLog() at ??:0
@ 0x7fa343ca34de google::LogMessage::Flush() at ??:0
@ 0x7fa343ca4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa346410104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa34640dd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fa34bbfd1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fa34bbfd48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fa34bbfd594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fa34bbc3c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fa34bbcdad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa345b710d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa345b66954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa3469ca6db start_thread at ??:0
@ 0x7fa3427a161f clone at ??:0
W20260924 07:17:28.133366 27186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:28.133555 27186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:28.133586 27186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:28.134696 27186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:28.134732 27186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:28.134765 27186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:17:28.136188 27186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:38027
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=41875
--tserver_master_addrs=127.10.130.254:39723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:28.136404 27186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:28.136612 27186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:28.142196 27226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:28.142190 27225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:28.142217 27228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:28.142421 27186 server_base.cc:1034] running on GCE node
I20260924 07:17:28.142588 27186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:28.142805 27186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:28.143946 27186 hybrid_clock.cc:648] HybridClock initialized: now 1790234248143931 us; error 30 us; skew 500 ppm
Sep 24 07:17:28 dist-test-slave-7q7m krb5kdc[26050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234248, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:28.146289 27186 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:17:28.146698 27186 webserver.cc:460] Webserver started at http://127.10.130.193:41875/ using document root <none> and password file <none>
I20260924 07:17:28.146880 27186 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:28.146925 27186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:28.148233 27186 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:17:28.148993 27235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:28.149168 27186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:17:28.149248 27186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "a06b3282a1ac4806950fbc48b9c0c1c8"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "2587d2694dc7bd991518eba1ff310f21"
server_key_iv: "a7f522e1b98c16198275e50833e81851"
server_key_version: "encryptionkey@0"
I20260924 07:17:28.149582 27186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:28.158907 27186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:28.160840 27186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:28.160964 27186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:28.161158 27186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:28.161602 27242 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 07:17:28.163414 27186 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 07:17:28.163455 27186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 07:17:28.163484 27186 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 07:17:28.164119 27186 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 07:17:28.164150 27186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260924 07:17:28.164224 27242 tablet_bootstrap.cc:492] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8: Bootstrap starting.
I20260924 07:17:28.166015 27242 log.cc:824] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:28.168890 27242 tablet_bootstrap.cc:492] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 07:17:28.169183 27242 tablet_bootstrap.cc:492] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8: Bootstrap complete.
I20260924 07:17:28.169327 27242 ts_tablet_manager.cc:1397] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260924 07:17:28.170557 27186 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:38027
I20260924 07:17:28.170889 27186 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:17:28.171077 27242 raft_consensus.cc:348] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:28.171368 27242 raft_consensus.cc:729] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a06b3282a1ac4806950fbc48b9c0c1c8, State: Initialized, Role: FOLLOWER
I20260924 07:17:28.171432 27350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:38027 every 8 connection(s)
I20260924 07:17:28.171617 27242 consensus_queue.cc:260] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:28.171840 27242 ts_tablet_manager.cc:1428] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 07:17:28.173911 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 27186
I20260924 07:17:28.174196 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:36525
--local_ip_for_outbound_sockets=127.10.130.194
--tserver_master_addrs=127.10.130.254:39723
--webserver_port=38725
--webserver_interface=127.10.130.194
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:28.234764 27354 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:17:28.171693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:47545)
0924 07:17:28.171855 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:28.171896 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:28.172062 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:28.234571 (+ 62509us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
W20260924 07:17:28.235019 27351 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:39723 (0 consecutive failures): Network error: Failed to ping master at 127.10.130.254:39723: Client connection negotiation failed: client connection to 127.10.130.254:39723: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 07:17:28.235358 27354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:28.235251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:43907)
0924 07:17:28.235301 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:28.235339 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 07:17:28.235623 27354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:28.235550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:48681)
0924 07:17:28.235587 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:28.235606 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 07:17:28.235719 27351 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 07:17:28.252234 27355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:28.252408 27355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:28.252435 27355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:28.253567 27355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:28.253607 27355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:28.253640 27355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:17:28.254978 27355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:36525
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=38725
--tserver_master_addrs=127.10.130.254:39723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:28.255203 27355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:28.255385 27355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:28.259886 27363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:28.259898 27361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:28.259940 27355 server_base.cc:1034] running on GCE node
W20260924 07:17:28.259902 27360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:28.260299 27355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:28.260528 27355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:28.261685 27355 hybrid_clock.cc:648] HybridClock initialized: now 1790234248261671 us; error 27 us; skew 500 ppm
Sep 24 07:17:28 dist-test-slave-7q7m krb5kdc[26050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234248, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:28.264071 27355 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:17:28.264464 27355 webserver.cc:460] Webserver started at http://127.10.130.194:38725/ using document root <none> and password file <none>
I20260924 07:17:28.264667 27355 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:28.264709 27355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:28.265931 27355 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:17:28.266444 27370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:28.266587 27355 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:28.266686 27355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "8f8ae26193364c129e8ea00d8be67f84"
format_stamp: "Formatted at 2026-09-24 07:16:46 on dist-test-slave-7q7m"
server_key: "622d3b1bcb051fa5273459bf6338846c"
server_key_iv: "6a3b85c32d27853336b581f0c301afc6"
server_key_version: "encryptionkey@0"
I20260924 07:17:28.267005 27355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:28.276585 27355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:28.278489 27355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:28.278609 27355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:28.278816 27355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:28.279078 27355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:28.279109 27355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:28.279130 27355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:28.279153 27355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:28.284266 27355 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:36525
I20260924 07:17:28.284339 27483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:36525 every 8 connection(s)
I20260924 07:17:28.284602 27355 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:17:28.285502 27486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:28.285186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:53105)
0924 07:17:28.285313 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:28.285373 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
W20260924 07:17:28.285879 27484 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:39723 (0 consecutive failures): Network error: Failed to ping master at 127.10.130.254:39723: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
I20260924 07:17:28.286113 27486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:28.286026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:49493)
0924 07:17:28.286068 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:28.286093 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260924 07:17:28.286317 27486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:28.286267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:50495)
0924 07:17:28.286287 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:28.286305 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260924 07:17:28.286393 27484 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 07:17:28.288730 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 27355
I20260924 07:17:28.288957 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:39855
--local_ip_for_outbound_sockets=127.10.130.195
--tserver_master_addrs=127.10.130.254:39723
--webserver_port=43945
--webserver_interface=127.10.130.195
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:33999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:28.366286 27487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:28.366477 27487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:28.366497 27487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:28.367605 27487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:28.367645 27487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:28.367673 27487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:17:28.369096 27487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:33999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:39855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=43945
--tserver_master_addrs=127.10.130.254:39723
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:28.369314 27487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:28.369535 27487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:28.374145 27493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:28.374155 27495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:28.374151 27492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:28.374293 27487 server_base.cc:1034] running on GCE node
I20260924 07:17:28.374534 27487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:28.374758 27487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:28.375900 27487 hybrid_clock.cc:648] HybridClock initialized: now 1790234248375884 us; error 26 us; skew 500 ppm
Sep 24 07:17:28 dist-test-slave-7q7m krb5kdc[26050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234248, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:28.378229 27487 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:17:28.378614 27487 webserver.cc:460] Webserver started at http://127.10.130.195:43945/ using document root <none> and password file <none>
I20260924 07:17:28.378805 27487 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:28.378851 27487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:28.380088 27487 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:17:28.380641 27502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:28.380784 27487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:28.380862 27487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "f02e15281e8c458a98fc01a0c81e0945"
format_stamp: "Formatted at 2026-09-24 07:16:47 on dist-test-slave-7q7m"
server_key: "221e03abf06307dbdcddfeb20de4aada"
server_key_iv: "0be994f86dcc4c2f6dee6eba3f8a2f6e"
server_key_version: "encryptionkey@0"
I20260924 07:17:28.381163 27487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:28.395895 27487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:28.397586 27487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:28.397713 27487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:28.397908 27487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:28.398180 27487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:28.398213 27487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:28.398241 27487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:28.398265 27487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:28.403489 27487 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:39855
I20260924 07:17:28.403558 27615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:39855 every 8 connection(s)
I20260924 07:17:28.403843 27487 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:17:28.404791 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:28.404455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:45143)
0924 07:17:28.404587 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:28.404654 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
W20260924 07:17:28.405169 27616 heartbeater.cc:644] Failed to heartbeat to 127.10.130.254:39723 (0 consecutive failures): Network error: Failed to ping master at 127.10.130.254:39723: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
I20260924 07:17:28.405391 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:28.405314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:59761)
0924 07:17:28.405350 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:28.405374 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260924 07:17:28.405586 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:28.405530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:39621)
0924 07:17:28.405559 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:28.405575 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260924 07:17:28.405650 27616 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 07:17:28.413057 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 27487
I20260924 07:17:29.236297 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:29.236079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:44965)
0924 07:17:29.236199 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:29.236235 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260924 07:17:29.286929 27621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:29.286714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:38053)
0924 07:17:29.286847 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:29.286876 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":66,"threads_started":1}
I20260924 07:17:29.406164 27622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:29.405922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:51023)
0924 07:17:29.406051 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:29.406105 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":57,"threads_started":1}
I20260924 07:17:29.440727 27623 raft_consensus.cc:482] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:17:29.440857 27623 raft_consensus.cc:504] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:29.441171 27623 leader_election.cc:288] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195:39855), 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525)
Sep 24 07:17:29 dist-test-slave-7q7m krb5kdc[26050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234248, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.195@KRBTEST.COM
Sep 24 07:17:29 dist-test-slave-7q7m krb5kdc[26050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234248, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.194@KRBTEST.COM
I20260924 07:17:29.450973 27625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:29.441420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:38525 (local address 127.10.130.194:36525)
0924 07:17:29.441719 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:29.441727 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:29.441755 (+ 28us) server_negotiation.cc:407] Connection header received
0924 07:17:29.442178 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:29.442180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:29.442225 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:29.442345 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:29.443059 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:29.443963 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:29.445005 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:29.445398 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:29.445494 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:29.447903 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:29.447906 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:29.447907 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:29.447917 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:29.449391 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:29.449784 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:29.449786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:29.449787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:29.449862 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:29.450081 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:29.450082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:29.450082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:29.450354 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:29.450431 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:29.450738 (+ 307us) server_negotiation.cc:299] Negotiation successful
0924 07:17:29.450863 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":75,"threads_started":1}
I20260924 07:17:29.450974 27626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:29.441461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:54005 (local address 127.10.130.195:39855)
0924 07:17:29.441598 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:29.441611 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:29.441639 (+ 28us) server_negotiation.cc:407] Connection header received
0924 07:17:29.442057 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:29.442059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:29.442110 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:29.442232 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:29.443144 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:29.444066 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:29.444992 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:29.445390 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:29.445481 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:29.447622 (+ 2141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:29.447626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:29.447627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:29.447638 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:29.449281 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:29.449928 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:29.449930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:29.449930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:29.450003 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:29.450196 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:29.450197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:29.450197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:29.450483 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:29.450538 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:29.450738 (+ 200us) server_negotiation.cc:299] Negotiation successful
0924 07:17:29.450863 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":51,"threads_started":1}
I20260924 07:17:29.450985 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:29.441317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:36525 (local address 127.10.130.193:38525)
0924 07:17:29.441383 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:29.441395 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:29.441463 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:29.442343 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:29.442344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:29.442658 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:29.442903 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:29.442911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:29.444104 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:29.444105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:29.444919 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:29.444922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:29.445472 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:29.445473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:29.445581 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:29.446171 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:29.446175 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:29.447811 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:29.449458 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:29.449458 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:29.449459 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:29.449716 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:29.449911 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:29.449912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:29.449912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:29.450008 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:29.450413 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:29.450419 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:29.450529 (+ 110us) client_negotiation.cc:769] Sending connection context
0924 07:17:29.450683 (+ 154us) client_negotiation.cc:240] Negotiation successful
0924 07:17:29.450842 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":52}
I20260924 07:17:29.451373 27624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:29.441330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:39855 (local address 127.10.130.193:54005)
0924 07:17:29.441498 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:29.441507 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:29.441566 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:29.442288 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:29.442290 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:29.442652 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:29.443048 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:29.443051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:29.444155 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:29.444156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:29.444917 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:29.444919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:29.445464 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:29.445465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:29.445586 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:29.446099 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:29.446106 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:29.447489 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:29.449362 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:29.449369 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:29.449371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:29.449715 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:29.450062 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:29.450062 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:29.450063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:29.450137 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:29.450544 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:29.450545 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:29.450578 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 07:17:29.450683 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 07:17:29.450908 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":161,"thread_start_us":72,"threads_started":1}
W20260924 07:17:29.451812 27238 leader_election.cc:341] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525): Not found: Tablet not found: de450178831446498554a40bb9d09fe3
W20260924 07:17:29.452054 27237 leader_election.cc:341] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195:39855): Not found: Tablet not found: de450178831446498554a40bb9d09fe3
I20260924 07:17:29.452100 27237 leader_election.cc:302] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06b3282a1ac4806950fbc48b9c0c1c8; no voters: 8f8ae26193364c129e8ea00d8be67f84, f02e15281e8c458a98fc01a0c81e0945
I20260924 07:17:29.452203 27623 raft_consensus.cc:2726] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 07:17:30.236938 27627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:30.236712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:35981)
0924 07:17:30.236862 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:30.236891 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
I20260924 07:17:30.287531 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:30.287332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:60793)
0924 07:17:30.287450 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:30.287482 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
I20260924 07:17:30.406728 27629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:30.406552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:55149)
0924 07:17:30.406663 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:30.406696 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":63,"threads_started":1}
I20260924 07:17:31.155637 27630 raft_consensus.cc:482] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:17:31.155757 27630 raft_consensus.cc:504] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:31.155926 27630 leader_election.cc:288] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195:39855), 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525)
W20260924 07:17:31.156293 27238 leader_election.cc:341] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525): Not found: Tablet not found: de450178831446498554a40bb9d09fe3
W20260924 07:17:31.156361 27237 leader_election.cc:341] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195:39855): Not found: Tablet not found: de450178831446498554a40bb9d09fe3
I20260924 07:17:31.156445 27237 leader_election.cc:302] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06b3282a1ac4806950fbc48b9c0c1c8; no voters: 8f8ae26193364c129e8ea00d8be67f84, f02e15281e8c458a98fc01a0c81e0945
I20260924 07:17:31.156528 27630 raft_consensus.cc:2726] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 07:17:31.237632 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:31.237379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:38263)
0924 07:17:31.237541 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:31.237585 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
I20260924 07:17:31.288151 27632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:31.287924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:55935)
0924 07:17:31.288075 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:31.288110 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
I20260924 07:17:31.407281 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:31.407092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:43357)
0924 07:17:31.407213 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:31.407243 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
I20260924 07:17:32.238260 27634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:32.238050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:40529)
0924 07:17:32.238176 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:32.238211 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":71,"threads_started":1}
I20260924 07:17:32.288748 27635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:32.288539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:41397)
0924 07:17:32.288666 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:32.288705 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":60,"threads_started":1}
I20260924 07:17:32.407873 27636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:32.407646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:34365)
0924 07:17:32.407793 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:32.407835 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
I20260924 07:17:33.238978 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:33.238744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:57077)
0924 07:17:33.238888 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:33.238933 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":64,"threads_started":1}
I20260924 07:17:33.289381 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:33.289182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:53303)
0924 07:17:33.289301 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:33.289338 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260924 07:17:33.408469 27639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:33.408279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:56119)
0924 07:17:33.408402 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:33.408434 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":74,"threads_started":1}
I20260924 07:17:33.661491 27640 raft_consensus.cc:482] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:17:33.661608 27640 raft_consensus.cc:504] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:33.661767 27640 leader_election.cc:288] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195:39855), 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525)
W20260924 07:17:33.662123 27238 leader_election.cc:341] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525): Not found: Tablet not found: de450178831446498554a40bb9d09fe3
W20260924 07:17:33.662217 27237 leader_election.cc:341] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195:39855): Not found: Tablet not found: de450178831446498554a40bb9d09fe3
I20260924 07:17:33.662271 27237 leader_election.cc:302] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06b3282a1ac4806950fbc48b9c0c1c8; no voters: 8f8ae26193364c129e8ea00d8be67f84, f02e15281e8c458a98fc01a0c81e0945
I20260924 07:17:33.662350 27640 raft_consensus.cc:2726] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 07:17:34.239522 27641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:34.239335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:56461)
0924 07:17:34.239447 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:34.239477 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
I20260924 07:17:34.289891 27642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:34.289710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:37407)
0924 07:17:34.289816 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:34.289851 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260924 07:17:34.409107 27643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:34.408861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:52547)
0924 07:17:34.409003 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:34.409045 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":60,"threads_started":1}
I20260924 07:17:35.240235 27644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:35.240021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:48375)
0924 07:17:35.240157 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:35.240189 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260924 07:17:35.290472 27645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:35.290279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:37789)
0924 07:17:35.290404 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:35.290433 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260924 07:17:35.409817 27646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:35.409590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:47753)
0924 07:17:35.409735 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:35.409773 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260924 07:17:36.240831 27647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:36.240589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:34567)
0924 07:17:36.240742 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:36.240782 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":77,"threads_started":1}
I20260924 07:17:36.291003 27648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:36.290809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:48409)
0924 07:17:36.290931 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:36.290964 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260924 07:17:36.410475 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:36.410243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:33621)
0924 07:17:36.410380 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:36.410419 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260924 07:17:37.241518 27650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:37.241271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:55977)
0924 07:17:37.241426 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:37.241468 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260924 07:17:37.291606 27651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:37.291399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:45123)
0924 07:17:37.291525 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:37.291560 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260924 07:17:37.411119 27652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:37.410923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:39227)
0924 07:17:37.411035 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:37.411077 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
I20260924 07:17:37.719259 27653 raft_consensus.cc:482] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:17:37.719357 27653 raft_consensus.cc:504] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f02e15281e8c458a98fc01a0c81e0945" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 39855 } } peers { permanent_uuid: "8f8ae26193364c129e8ea00d8be67f84" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 36525 } } peers { permanent_uuid: "a06b3282a1ac4806950fbc48b9c0c1c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38027 } }
I20260924 07:17:37.719532 27653 leader_election.cc:288] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195:39855), 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525)
W20260924 07:17:37.719892 27238 leader_election.cc:341] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8f8ae26193364c129e8ea00d8be67f84 (127.10.130.194:36525): Not found: Tablet not found: de450178831446498554a40bb9d09fe3
W20260924 07:17:37.719996 27237 leader_election.cc:341] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f02e15281e8c458a98fc01a0c81e0945 (127.10.130.195:39855): Not found: Tablet not found: de450178831446498554a40bb9d09fe3
I20260924 07:17:37.720037 27237 leader_election.cc:302] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a06b3282a1ac4806950fbc48b9c0c1c8; no voters: 8f8ae26193364c129e8ea00d8be67f84, f02e15281e8c458a98fc01a0c81e0945
I20260924 07:17:37.720094 27653 raft_consensus.cc:2726] T de450178831446498554a40bb9d09fe3 P a06b3282a1ac4806950fbc48b9c0c1c8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 07:17:38.242187 27654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:38.241988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:52779)
0924 07:17:38.242102 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:38.242141 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260924 07:17:38.292309 27655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:38.292105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:34977)
0924 07:17:38.292232 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:38.292267 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260924 07:17:38.411659 27656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:38.411476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:39065)
0924 07:17:38.411585 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:38.411618 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":60,"threads_started":1}
I20260924 07:17:39.242813 27657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:39.242593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:40297)
0924 07:17:39.242731 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:39.242767 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260924 07:17:39.292861 27658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:39.292695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:38907)
0924 07:17:39.292796 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:39.292826 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":56,"threads_started":1}
I20260924 07:17:39.412199 27659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:39.412027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:54313)
0924 07:17:39.412129 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:39.412163 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260924 07:17:40.243450 27660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:40.243231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:48415)
0924 07:17:40.243367 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:40.243400 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260924 07:17:40.293423 27661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:40.293231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:57651)
0924 07:17:40.293350 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:40.293384 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260924 07:17:40.412781 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:40.412583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:36849)
0924 07:17:40.412702 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:40.412737 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260924 07:17:41.244161 27663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:41.243909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:50063)
0924 07:17:41.244070 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:41.244114 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
I20260924 07:17:41.294072 27664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:41.293854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:54259)
0924 07:17:41.293987 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:41.294024 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260924 07:17:41.413340 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:41.413134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:51025)
0924 07:17:41.413266 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:41.413300 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260924 07:17:42.244822 27666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:42.244568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:47589)
0924 07:17:42.244725 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:42.244765 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
I20260924 07:17:42.294687 27667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:42.294441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:60409)
0924 07:17:42.294569 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:42.294642 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260924 07:17:42.413905 27668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:42.413712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:46447)
0924 07:17:42.413834 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:42.413862 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260924 07:17:43.245548 27669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:43.245340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.193:46867)
0924 07:17:43.245472 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:43.245505 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260924 07:17:43.295344 27670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:43.295148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.194:45847)
0924 07:17:43.295269 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:43.295304 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
I20260924 07:17:43.414108 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 27186
I20260924 07:17:43.415051 27671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:43.414770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39723 (local address 127.10.130.195:60435)
0924 07:17:43.414971 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:43.415004 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.254:39723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":49,"threads_started":1}
I20260924 07:17:43.418124 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 27355
I20260924 07:17:43.422225 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 27487
2026-09-24T07:17:43Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (57967 ms)
[----------] 30 tests from SecurityITest (218461 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:17:43 dist-test-slave-7q7m krb5kdc[27674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:17:43 dist-test-slave-7q7m krb5kdc[27674](info): set up 2 sockets
Sep 24 07:17:43 dist-test-slave-7q7m krb5kdc[27674](info): commencing operation
krb5kdc: starting...
W20260924 07:17:44.367376 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:17:44 dist-test-slave-7q7m krb5kdc[27674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:17:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:17:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:17:44.474705 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:33593
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:33593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:44.552316 27687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:44.552508 27687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:44.552528 27687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:44.553645 27687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:44.553690 27687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:44.553702 27687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:44.553714 27687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:44.553723 27687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:44.555140 27687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:33593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:33593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:44.555361 27687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:44.555573 27687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:44.559878 27695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:44.559894 27692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:44.559893 27693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:44.560066 27687 server_base.cc:1034] running on GCE node
I20260924 07:17:44.560350 27687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:44.560633 27687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:44.561777 27687 hybrid_clock.cc:648] HybridClock initialized: now 1790234264561761 us; error 31 us; skew 500 ppm
Sep 24 07:17:44 dist-test-slave-7q7m krb5kdc[27674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234264, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:44.568358 27687 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:17:44.568789 27687 webserver.cc:460] Webserver started at http://127.10.130.254:45371/ using document root <none> and password file <none>
I20260924 07:17:44.568987 27687 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:44.569036 27687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:44.569156 27687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:44.570181 27687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "2eb93e524fc8470389d0428c80ebc271"
format_stamp: "Formatted at 2026-09-24 07:17:44 on dist-test-slave-7q7m"
server_key: "90df500567fc6c8831712cfab4dfac83"
server_key_iv: "c998e40c2ae90ffa853ba8b2d6c2cd88"
server_key_version: "encryptionkey@0"
I20260924 07:17:44.570511 27687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "2eb93e524fc8470389d0428c80ebc271"
format_stamp: "Formatted at 2026-09-24 07:17:44 on dist-test-slave-7q7m"
server_key: "90df500567fc6c8831712cfab4dfac83"
server_key_iv: "c998e40c2ae90ffa853ba8b2d6c2cd88"
server_key_version: "encryptionkey@0"
I20260924 07:17:44.572072 27687 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:17:44.572703 27702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:44.572850 27687 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:44.572932 27687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "2eb93e524fc8470389d0428c80ebc271"
format_stamp: "Formatted at 2026-09-24 07:17:44 on dist-test-slave-7q7m"
server_key: "90df500567fc6c8831712cfab4dfac83"
server_key_iv: "c998e40c2ae90ffa853ba8b2d6c2cd88"
server_key_version: "encryptionkey@0"
I20260924 07:17:44.572997 27687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:44.588883 27687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:44.590982 27687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:44.591120 27687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:44.594938 27687 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:33593
I20260924 07:17:44.594972 27754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:33593 every 8 connection(s)
I20260924 07:17:44.595305 27687 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:44.596154 27755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:44.598377 27755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271: Bootstrap starting.
I20260924 07:17:44.598812 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 27687
I20260924 07:17:44.598920 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:17:44.599021 27755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:44.599109 10763 external_mini_cluster.cc:1428] Setting key baf57a2f4dd646a21b5b06d09ef586a9
I20260924 07:17:44.599331 27755 log.cc:824] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:44.600008 27755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271: No bootstrap required, opened a new log
I20260924 07:17:44.601913 27755 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb93e524fc8470389d0428c80ebc271" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33593 } }
I20260924 07:17:44.602044 27755 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:44.602065 27755 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eb93e524fc8470389d0428c80ebc271, State: Initialized, Role: FOLLOWER
I20260924 07:17:44.602202 27755 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb93e524fc8470389d0428c80ebc271" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33593 } }
I20260924 07:17:44.602277 27755 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:44.602308 27755 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:44.602353 27755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:44.602833 27755 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb93e524fc8470389d0428c80ebc271" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33593 } }
I20260924 07:17:44.602922 27755 leader_election.cc:302] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2eb93e524fc8470389d0428c80ebc271; no voters:
I20260924 07:17:44.603080 27755 leader_election.cc:288] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:17:44.603113 27760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:44.603217 27760 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [term 1 LEADER]: Becoming Leader. State: Replica: 2eb93e524fc8470389d0428c80ebc271, State: Running, Role: LEADER
I20260924 07:17:44.603349 27755 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:44.603346 27760 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb93e524fc8470389d0428c80ebc271" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33593 } }
I20260924 07:17:44.603778 27761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2eb93e524fc8470389d0428c80ebc271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb93e524fc8470389d0428c80ebc271" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33593 } } }
I20260924 07:17:44.604031 27761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:44.603945 27762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2eb93e524fc8470389d0428c80ebc271. Latest consensus state: current_term: 1 leader_uuid: "2eb93e524fc8470389d0428c80ebc271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2eb93e524fc8470389d0428c80ebc271" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33593 } } }
I20260924 07:17:44.604435 27762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:44.604666 27769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:44.605160 27769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 07:17:44 dist-test-slave-7q7m krb5kdc[27674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:44.607075 27769 catalog_manager.cc:1347] Generated new cluster ID: 306dd860e1b54823810b89c1b6e19a09
I20260924 07:17:44.607129 27769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:17:44.610383 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:44.599595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38222 (local address 127.10.130.254:33593)
0924 07:17:44.599743 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:44.599753 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:44.599791 (+ 38us) server_negotiation.cc:407] Connection header received
0924 07:17:44.600327 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:44.600336 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:44.600388 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:44.600520 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:44.601167 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:44.602095 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:44.602927 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:44.603511 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:44.603621 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:44.606384 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:44.606388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:44.606388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:44.606401 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:44.608430 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:44.608838 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:44.608841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:44.608842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:44.608945 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:44.609148 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:44.609149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:44.609149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:44.609451 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:44.609605 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:44.610039 (+ 434us) server_negotiation.cc:299] Negotiation successful
0924 07:17:44.610176 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
I20260924 07:17:44.613044 27769 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:17:44.613758 27769 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:44.618443 27769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271: Generated new TSK 0
I20260924 07:17:44.618618 27769 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:17:44.641005 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:33593
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:44.718533 27781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:44.718740 27781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:44.718771 27781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:44.719902 27781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:44.719942 27781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:44.719995 27781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:17:44.721438 27781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:33593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:44.721680 27781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:44.721927 27781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:44.727479 27789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:44.727603 27781 server_base.cc:1034] running on GCE node
W20260924 07:17:44.727519 27787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:44.727682 27786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:44.727905 27781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:44.728183 27781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:44.729328 27781 hybrid_clock.cc:648] HybridClock initialized: now 1790234264729317 us; error 25 us; skew 500 ppm
Sep 24 07:17:44 dist-test-slave-7q7m krb5kdc[27674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234264, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:44.731688 27781 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:17:44.732164 27781 webserver.cc:460] Webserver started at http://127.10.130.193:39103/ using document root <none> and password file <none>
I20260924 07:17:44.732359 27781 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:44.732396 27781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:44.732486 27781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:44.733470 27781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc"
format_stamp: "Formatted at 2026-09-24 07:17:44 on dist-test-slave-7q7m"
server_key: "8cf6cb69637fe8af61a7f058af9608be"
server_key_iv: "3aa7999bcc8317fe021a957b6e95c38d"
server_key_version: "encryptionkey@0"
I20260924 07:17:44.733763 27781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc"
format_stamp: "Formatted at 2026-09-24 07:17:44 on dist-test-slave-7q7m"
server_key: "8cf6cb69637fe8af61a7f058af9608be"
server_key_iv: "3aa7999bcc8317fe021a957b6e95c38d"
server_key_version: "encryptionkey@0"
I20260924 07:17:44.734961 27781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:17:44.735661 27796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:44.735797 27781 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:44.735877 27781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc"
format_stamp: "Formatted at 2026-09-24 07:17:44 on dist-test-slave-7q7m"
server_key: "8cf6cb69637fe8af61a7f058af9608be"
server_key_iv: "3aa7999bcc8317fe021a957b6e95c38d"
server_key_version: "encryptionkey@0"
I20260924 07:17:44.735936 27781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:44.757045 27781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:44.759124 27781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:44.759253 27781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:44.759466 27781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:44.759752 27781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:44.759785 27781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:44.759814 27781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:44.759836 27781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:44.765187 27781 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:46085
I20260924 07:17:44.765244 27909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:46085 every 8 connection(s)
I20260924 07:17:44.765552 27781 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
Sep 24 07:17:44 dist-test-slave-7q7m krb5kdc[27674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234264, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:44.773911 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:44.766212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:54221 (local address 127.10.130.254:33593)
0924 07:17:44.766289 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:44.766291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:44.766440 (+ 149us) server_negotiation.cc:407] Connection header received
0924 07:17:44.766899 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:44.766900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:44.766909 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:44.766957 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:44.767806 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:44.768351 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:44.769265 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:44.769457 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:44.769504 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:44.771525 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:44.771526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:44.771526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:44.771533 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:44.772741 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:44.773088 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:44.773089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:44.773090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:44.773156 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:44.773422 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:44.773422 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:44.773422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:44.773521 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:44.773569 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:44.773777 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 07:17:44.773845 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 07:17:44.774084 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:44.766181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:33593 (local address 127.10.130.193:54221)
0924 07:17:44.766326 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:44.766363 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:44.766825 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:44.767014 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:44.767017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:44.767554 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:44.767742 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:44.767747 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:44.768419 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:44.768420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:44.769203 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:44.769205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:44.769510 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:44.769511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:44.769602 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:44.770039 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:44.770047 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:44.771320 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:44.772807 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:44.772812 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:44.772814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:44.773008 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:44.773245 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:44.773246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:44.773246 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:44.773367 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:44.773566 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:44.773570 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:44.773683 (+ 113us) client_negotiation.cc:769] Sending connection context
0924 07:17:44.773761 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 07:17:44.773876 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260924 07:17:44.774546 27910 heartbeater.cc:347] Connected to a master server at 127.10.130.254:33593
I20260924 07:17:44.774634 27910 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:44.774814 27910 heartbeater.cc:511] Master 127.10.130.254:33593 requested a full tablet report, sending...
I20260924 07:17:44.775291 27719 ts_manager.cc:194] Registered new tserver with Master: 37a3f5de9a4f4cf19e73f8ebb802acdc (127.10.130.193:46085)
I20260924 07:17:44.775625 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 27781
I20260924 07:17:44.775717 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:17:44.775887 10763 external_mini_cluster.cc:1428] Setting key a6dce1434955c2854b8dda7285bc2294
I20260924 07:17:44.776010 27719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:54221
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:17:44.796655 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:33593
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:44.874986 27915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:44.875182 27915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:44.875203 27915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:44.876396 27915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:44.876434 27915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:44.876466 27915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:17:44.877831 27915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:33593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:44.877998 27915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:44.878170 27915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:44.882711 27923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:44.882686 27915 server_base.cc:1034] running on GCE node
W20260924 07:17:44.882649 27921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:44.882659 27920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:44.883052 27915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:44.883281 27915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:44.884414 27915 hybrid_clock.cc:648] HybridClock initialized: now 1790234264884402 us; error 30 us; skew 500 ppm
Sep 24 07:17:44 dist-test-slave-7q7m krb5kdc[27674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234264, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:44.886612 27915 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:17:44.886987 27915 webserver.cc:460] Webserver started at http://127.10.130.194:34437/ using document root <none> and password file <none>
I20260924 07:17:44.887176 27915 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:44.887223 27915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:44.887338 27915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:44.888348 27915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "3de25d8e0bde44a99473e85d85f9be02"
format_stamp: "Formatted at 2026-09-24 07:17:44 on dist-test-slave-7q7m"
server_key: "178521e979bb85fbd089c4beeb5e134f"
server_key_iv: "cd60e98ca05ebb88550e4d12d0da1e9a"
server_key_version: "encryptionkey@0"
I20260924 07:17:44.888659 27915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "3de25d8e0bde44a99473e85d85f9be02"
format_stamp: "Formatted at 2026-09-24 07:17:44 on dist-test-slave-7q7m"
server_key: "178521e979bb85fbd089c4beeb5e134f"
server_key_iv: "cd60e98ca05ebb88550e4d12d0da1e9a"
server_key_version: "encryptionkey@0"
I20260924 07:17:44.889883 27915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:17:44.890609 27930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:44.890745 27915 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:44.890820 27915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "3de25d8e0bde44a99473e85d85f9be02"
format_stamp: "Formatted at 2026-09-24 07:17:44 on dist-test-slave-7q7m"
server_key: "178521e979bb85fbd089c4beeb5e134f"
server_key_iv: "cd60e98ca05ebb88550e4d12d0da1e9a"
server_key_version: "encryptionkey@0"
I20260924 07:17:44.890877 27915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:44.902872 27915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:44.904876 27915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:44.905001 27915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:44.905217 27915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:44.905501 27915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:44.905532 27915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:44.905560 27915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:44.905578 27915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:44.910800 27915 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:46105
I20260924 07:17:44.910857 28043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:46105 every 8 connection(s)
I20260924 07:17:44.911134 27915 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
Sep 24 07:17:44 dist-test-slave-7q7m krb5kdc[27674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234264, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:44.919026 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:44.911763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:49727 (local address 127.10.130.254:33593)
0924 07:17:44.911815 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:44.911817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:44.912012 (+ 195us) server_negotiation.cc:407] Connection header received
0924 07:17:44.912496 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:44.912497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:44.912509 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:44.912555 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:44.913446 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:44.913939 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:44.914898 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:44.915083 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:44.915144 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:44.917018 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:44.917019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:44.917020 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:44.917026 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:44.918043 (+ 1017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:44.918375 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:44.918375 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:44.918376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:44.918420 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:44.918587 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:44.918587 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:44.918587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:44.918675 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:44.918719 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:44.918911 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 07:17:44.918965 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 07:17:44.919183 28046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:44.911748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:33593 (local address 127.10.130.194:49727)
0924 07:17:44.911901 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:44.911933 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:44.912406 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:44.912644 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:44.912648 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:44.913172 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:44.913354 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:44.913359 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:44.914016 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:44.914017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:44.914819 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:44.914821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:44.915157 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:44.915157 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:44.915280 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:44.915667 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:44.915674 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:44.916944 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:44.918115 (+ 1171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:44.918119 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:44.918121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:44.918321 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:44.918463 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:44.918463 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:44.918463 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:44.918547 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:44.918714 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:44.918717 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:44.918822 (+ 105us) client_negotiation.cc:769] Sending connection context
0924 07:17:44.918895 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 07:17:44.919003 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
I20260924 07:17:44.919620 28044 heartbeater.cc:347] Connected to a master server at 127.10.130.254:33593
I20260924 07:17:44.919705 28044 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:44.919862 28044 heartbeater.cc:511] Master 127.10.130.254:33593 requested a full tablet report, sending...
I20260924 07:17:44.920224 27718 ts_manager.cc:194] Registered new tserver with Master: 3de25d8e0bde44a99473e85d85f9be02 (127.10.130.194:46105)
I20260924 07:17:44.920580 27718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:49727
I20260924 07:17:44.920820 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 27915
I20260924 07:17:44.920938 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:17:44.921159 10763 external_mini_cluster.cc:1428] Setting key 3daf0bc35391afd1faa3ee94c1743965
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:17:44.942678 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:33593
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:35927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:45.019311 28049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:45.019502 28049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:45.019531 28049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:45.020669 28049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:45.020710 28049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:45.020744 28049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:17:45.022128 28049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:35927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:33593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:45.022365 28049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:45.022558 28049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:45.027442 28054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:45.027446 28055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:45.028069 28057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:45.028415 28049 server_base.cc:1034] running on GCE node
I20260924 07:17:45.028597 28049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:45.028812 28049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:45.029949 28049 hybrid_clock.cc:648] HybridClock initialized: now 1790234265029936 us; error 28 us; skew 500 ppm
Sep 24 07:17:45 dist-test-slave-7q7m krb5kdc[27674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234265, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:45.032241 28049 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:17:45.032671 28049 webserver.cc:460] Webserver started at http://127.10.130.195:34631/ using document root <none> and password file <none>
I20260924 07:17:45.032862 28049 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:45.032908 28049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:45.033030 28049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:45.034053 28049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "8fcac3dd8537d5172b5eab6a518ad8f9"
server_key_iv: "317027d88f70c81afbf50b4757704ec8"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.034384 28049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "8fcac3dd8537d5172b5eab6a518ad8f9"
server_key_iv: "317027d88f70c81afbf50b4757704ec8"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.035820 28049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:17:45.036558 28064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.036693 28049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.036773 28049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "8fcac3dd8537d5172b5eab6a518ad8f9"
server_key_iv: "317027d88f70c81afbf50b4757704ec8"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.036839 28049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:45.052374 28049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:45.054127 28049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:45.054273 28049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:45.054488 28049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:45.054774 28049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:45.054807 28049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.054836 28049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:45.054852 28049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.060267 28049 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:42703
I20260924 07:17:45.060316 28177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:42703 every 8 connection(s)
I20260924 07:17:45.060611 28049 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
Sep 24 07:17:45 dist-test-slave-7q7m krb5kdc[27674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234265, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:45.066792 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 28049
I20260924 07:17:45.066905 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:17:45.067072 10763 external_mini_cluster.cc:1428] Setting key a5e0e9f7af1dff3d017481407ba0f2d3
I20260924 07:17:45.069159 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.061288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:33215 (local address 127.10.130.254:33593)
0924 07:17:45.061403 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.061405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.061512 (+ 107us) server_negotiation.cc:407] Connection header received
0924 07:17:45.061977 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.061978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.061987 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.062035 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:45.062900 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.063404 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.064327 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.064505 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.064547 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.066467 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:45.066468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:45.066468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:45.066474 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:45.067924 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:45.068296 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:45.068297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:45.068297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:45.068355 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:45.068555 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:45.068556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:45.068557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:45.068679 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:45.068735 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.068978 (+ 243us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.069057 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 07:17:45.069316 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.061246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:33593 (local address 127.10.130.195:33215)
0924 07:17:45.061397 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.061442 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.061904 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.062091 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.062095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.062623 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:45.062832 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.062837 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.063460 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.063460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.064272 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.064275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.064554 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.064555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.064663 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.065117 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:45.065126 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:45.066381 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:45.068013 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:45.068020 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:45.068023 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:45.068232 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:45.068407 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:45.068408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:45.068408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:45.068502 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:45.068733 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:45.068739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:45.068865 (+ 126us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.068952 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.069106 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260924 07:17:45.069706 28178 heartbeater.cc:347] Connected to a master server at 127.10.130.254:33593
I20260924 07:17:45.069787 28178 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:45.069926 28178 heartbeater.cc:511] Master 127.10.130.254:33593 requested a full tablet report, sending...
I20260924 07:17:45.070245 27718 ts_manager.cc:194] Registered new tserver with Master: 5e1b6db9514e491e9f02b7f9d92aa6f6 (127.10.130.195:42703)
I20260924 07:17:45.070634 27718 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:33215
I20260924 07:17:45.078644 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:17:45.085049 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.079886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.10.130.254:33593)
0924 07:17:45.079929 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.079931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.080038 (+ 107us) server_negotiation.cc:407] Connection header received
0924 07:17:45.080095 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.080096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.080104 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.080157 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:45.080650 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.081129 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.081805 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.081977 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.082016 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.083033 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:45.083034 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:45.083034 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:45.083040 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:45.084249 (+ 1209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:45.084494 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:45.084495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:45.084495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:45.084539 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:45.084653 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:45.084653 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:45.084654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:45.084782 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:45.084852 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.084925 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.084985 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 07:17:45.086449 27718 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:17:45.087913 27718 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:17:45.093510 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.089868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:46085 (local address 127.0.0.1:45232)
0924 07:17:45.090190 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.090229 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.090300 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.090625 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.090628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.090630 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:45.090704 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.090706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.091556 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.091556 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.092429 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.092432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.093178 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.093178 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.093277 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.093291 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.093386 (+ 95us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.093439 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":52,"threads_started":1}
I20260924 07:17:45.093531 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.089989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.10.130.193:46085)
0924 07:17:45.090183 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.090187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.090289 (+ 102us) server_negotiation.cc:407] Connection header received
0924 07:17:45.090417 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.090419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.090437 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.090491 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:45.090783 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.091469 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.092526 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.093050 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.093086 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.093122 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.093423 (+ 301us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.093467 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":100,"threads_started":1}
I20260924 07:17:45.094337 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.090056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:46105 (local address 127.0.0.1:43216)
0924 07:17:45.090419 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.090427 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.090473 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.090729 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.090729 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.090731 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:45.090763 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.090763 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.092024 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.092025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.093261 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.093262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.094094 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.094095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.094156 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.094161 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.094225 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.094266 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":34,"thread_start_us":58,"threads_started":1}
I20260924 07:17:45.094264 27844 tablet_service.cc:1467] Processing CreateTablet for tablet 3f8ac5c12b514a0b899d7afe617558ca (DEFAULT_TABLE table=test-table [id=c7fa9223e67840148c0683ab2dddbf1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:45.094412 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.090183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43216 (local address 127.10.130.194:46105)
0924 07:17:45.090360 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.090365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.090531 (+ 166us) server_negotiation.cc:407] Connection header received
0924 07:17:45.090574 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.090576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.090591 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.090654 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:45.090823 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.091896 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.093329 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.094013 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.094085 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.094136 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.094241 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.094335 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":85,"threads_started":1}
I20260924 07:17:45.094609 27844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f8ac5c12b514a0b899d7afe617558ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:45.094698 28189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.089870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:42703 (local address 127.0.0.1:59638)
0924 07:17:45.090353 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.090363 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.090423 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.090665 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.090665 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.090668 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:45.090720 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.090722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.092063 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.092064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.093259 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.093262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.094400 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.094401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.094547 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.094551 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.094603 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.094637 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":57,"thread_start_us":53,"threads_started":1}
I20260924 07:17:45.095027 28191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.090194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59638 (local address 127.10.130.195:42703)
0924 07:17:45.090372 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.090376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.090420 (+ 44us) server_negotiation.cc:407] Connection header received
0924 07:17:45.090490 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.090493 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.090508 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.090577 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:45.090793 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.091944 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.093633 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.094257 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.094314 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.094367 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.094632 (+ 265us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.094953 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":75,"threads_started":1}
I20260924 07:17:45.095338 27978 tablet_service.cc:1467] Processing CreateTablet for tablet 3f8ac5c12b514a0b899d7afe617558ca (DEFAULT_TABLE table=test-table [id=c7fa9223e67840148c0683ab2dddbf1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:45.095674 27978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f8ac5c12b514a0b899d7afe617558ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:45.095638 28112 tablet_service.cc:1467] Processing CreateTablet for tablet 3f8ac5c12b514a0b899d7afe617558ca (DEFAULT_TABLE table=test-table [id=c7fa9223e67840148c0683ab2dddbf1f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:45.095948 28112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f8ac5c12b514a0b899d7afe617558ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:45.097350 28192 tablet_bootstrap.cc:492] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc: Bootstrap starting.
I20260924 07:17:45.098044 28192 tablet_bootstrap.cc:654] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:45.098495 28192 log.cc:824] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:45.098865 28195 tablet_bootstrap.cc:492] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6: Bootstrap starting.
I20260924 07:17:45.099200 28192 tablet_bootstrap.cc:492] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc: No bootstrap required, opened a new log
I20260924 07:17:45.099255 28192 ts_tablet_manager.cc:1397] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:17:45.099478 28195 tablet_bootstrap.cc:654] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:45.100035 28195 log.cc:824] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:45.100375 28194 tablet_bootstrap.cc:492] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02: Bootstrap starting.
I20260924 07:17:45.100719 28195 tablet_bootstrap.cc:492] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6: No bootstrap required, opened a new log
I20260924 07:17:45.100778 28195 ts_tablet_manager.cc:1397] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:17:45.100970 28194 tablet_bootstrap.cc:654] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:45.101261 28192 raft_consensus.cc:348] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 42703 } } peers { permanent_uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46085 } } peers { permanent_uuid: "3de25d8e0bde44a99473e85d85f9be02" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46105 } }
I20260924 07:17:45.101464 28192 raft_consensus.cc:374] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:45.101487 28194 log.cc:824] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:45.101506 28192 raft_consensus.cc:729] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37a3f5de9a4f4cf19e73f8ebb802acdc, State: Initialized, Role: FOLLOWER
I20260924 07:17:45.101657 28192 consensus_queue.cc:260] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 42703 } } peers { permanent_uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46085 } } peers { permanent_uuid: "3de25d8e0bde44a99473e85d85f9be02" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46105 } }
I20260924 07:17:45.101925 28192 ts_tablet_manager.cc:1428] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:17:45.102030 27910 heartbeater.cc:503] Master 127.10.130.254:33593 was elected leader, sending a full tablet report...
I20260924 07:17:45.102218 28194 tablet_bootstrap.cc:492] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02: No bootstrap required, opened a new log
I20260924 07:17:45.102277 28194 ts_tablet_manager.cc:1397] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:17:45.102497 28195 raft_consensus.cc:348] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 42703 } } peers { permanent_uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46085 } } peers { permanent_uuid: "3de25d8e0bde44a99473e85d85f9be02" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46105 } }
I20260924 07:17:45.102623 28195 raft_consensus.cc:374] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:45.102667 28195 raft_consensus.cc:729] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e1b6db9514e491e9f02b7f9d92aa6f6, State: Initialized, Role: FOLLOWER
I20260924 07:17:45.102808 28195 consensus_queue.cc:260] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 42703 } } peers { permanent_uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46085 } } peers { permanent_uuid: "3de25d8e0bde44a99473e85d85f9be02" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46105 } }
I20260924 07:17:45.103055 28195 ts_tablet_manager.cc:1428] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 07:17:45.103104 28178 heartbeater.cc:503] Master 127.10.130.254:33593 was elected leader, sending a full tablet report...
I20260924 07:17:45.104038 28194 raft_consensus.cc:348] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 42703 } } peers { permanent_uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46085 } } peers { permanent_uuid: "3de25d8e0bde44a99473e85d85f9be02" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46105 } }
I20260924 07:17:45.104214 28194 raft_consensus.cc:374] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:45.104259 28194 raft_consensus.cc:729] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3de25d8e0bde44a99473e85d85f9be02, State: Initialized, Role: FOLLOWER
I20260924 07:17:45.104367 28194 consensus_queue.cc:260] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 42703 } } peers { permanent_uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46085 } } peers { permanent_uuid: "3de25d8e0bde44a99473e85d85f9be02" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46105 } }
I20260924 07:17:45.104619 28194 ts_tablet_manager.cc:1428] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 07:17:45.104645 28044 heartbeater.cc:503] Master 127.10.130.254:33593 was elected leader, sending a full tablet report...
W20260924 07:17:45.161677 28045 tablet.cc:2369] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:17:45.231292 28199 raft_consensus.cc:482] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:17:45.231411 28199 raft_consensus.cc:504] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 42703 } } peers { permanent_uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46085 } } peers { permanent_uuid: "3de25d8e0bde44a99473e85d85f9be02" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46105 } }
I20260924 07:17:45.231734 28199 leader_election.cc:288] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37a3f5de9a4f4cf19e73f8ebb802acdc (127.10.130.193:46085), 3de25d8e0bde44a99473e85d85f9be02 (127.10.130.194:46105)
I20260924 07:17:45.235302 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.232027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:37469 (local address 127.10.130.193:46085)
0924 07:17:45.232086 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.232088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.232100 (+ 12us) server_negotiation.cc:407] Connection header received
0924 07:17:45.232186 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.232187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.232197 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.232244 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:45.232418 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.233030 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.234368 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.234965 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.235039 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.235047 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.235185 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.235227 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 07:17:45.235302 28190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.232203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:34489 (local address 127.10.130.194:46105)
0924 07:17:45.232263 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.232265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.232321 (+ 56us) server_negotiation.cc:407] Connection header received
0924 07:17:45.232396 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.232397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.232406 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.232450 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:45.232600 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.233194 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.234405 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.234965 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.235039 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.235047 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.235198 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.235235 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 07:17:45.235312 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.231945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:46085 (local address 127.10.130.195:37469)
0924 07:17:45.232024 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.232037 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.232097 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.232256 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.232257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.232259 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:45.232348 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.232351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.233098 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.233099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.234277 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.234280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.235038 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.235039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.235124 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.235127 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.235175 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.235219 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":31}
I20260924 07:17:45.235318 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.232083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:46105 (local address 127.10.130.195:34489)
0924 07:17:45.232265 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.232275 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.232334 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.232454 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.232454 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.232456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:45.232513 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.232514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.233260 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.233260 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.234330 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.234332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.235038 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.235039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.235119 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.235123 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.235186 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.235219 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":4,"thread_start_us":123,"threads_started":1}
I20260924 07:17:45.235869 27998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f8ac5c12b514a0b899d7afe617558ca" candidate_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3de25d8e0bde44a99473e85d85f9be02" is_pre_election: true
I20260924 07:17:45.235877 27864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f8ac5c12b514a0b899d7afe617558ca" candidate_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc" is_pre_election: true
I20260924 07:17:45.236035 27998 raft_consensus.cc:2454] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e1b6db9514e491e9f02b7f9d92aa6f6 in term 0.
I20260924 07:17:45.236037 27864 raft_consensus.cc:2454] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e1b6db9514e491e9f02b7f9d92aa6f6 in term 0.
I20260924 07:17:45.236229 28065 leader_election.cc:302] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3de25d8e0bde44a99473e85d85f9be02, 5e1b6db9514e491e9f02b7f9d92aa6f6; no voters:
I20260924 07:17:45.236359 28199 raft_consensus.cc:2781] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:17:45.236423 28199 raft_consensus.cc:482] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:17:45.236438 28199 raft_consensus.cc:3037] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:45.237092 28199 raft_consensus.cc:504] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 42703 } } peers { permanent_uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46085 } } peers { permanent_uuid: "3de25d8e0bde44a99473e85d85f9be02" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46105 } }
I20260924 07:17:45.237193 28199 leader_election.cc:288] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [CANDIDATE]: Term 1 election: Requested vote from peers 37a3f5de9a4f4cf19e73f8ebb802acdc (127.10.130.193:46085), 3de25d8e0bde44a99473e85d85f9be02 (127.10.130.194:46105)
I20260924 07:17:45.237381 27864 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f8ac5c12b514a0b899d7afe617558ca" candidate_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc"
I20260924 07:17:45.237385 27998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f8ac5c12b514a0b899d7afe617558ca" candidate_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3de25d8e0bde44a99473e85d85f9be02"
I20260924 07:17:45.237443 27864 raft_consensus.cc:3037] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:45.237455 27998 raft_consensus.cc:3037] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:45.238137 27864 raft_consensus.cc:2454] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e1b6db9514e491e9f02b7f9d92aa6f6 in term 1.
I20260924 07:17:45.238180 27998 raft_consensus.cc:2454] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e1b6db9514e491e9f02b7f9d92aa6f6 in term 1.
I20260924 07:17:45.238282 28065 leader_election.cc:302] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37a3f5de9a4f4cf19e73f8ebb802acdc, 5e1b6db9514e491e9f02b7f9d92aa6f6; no voters:
I20260924 07:17:45.238360 28199 raft_consensus.cc:2781] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:45.238493 28199 raft_consensus.cc:686] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [term 1 LEADER]: Becoming Leader. State: Replica: 5e1b6db9514e491e9f02b7f9d92aa6f6, State: Running, Role: LEADER
I20260924 07:17:45.238572 28199 consensus_queue.cc:237] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 42703 } } peers { permanent_uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46085 } } peers { permanent_uuid: "3de25d8e0bde44a99473e85d85f9be02" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46105 } }
I20260924 07:17:45.239392 27718 catalog_manager.cc:5495] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e1b6db9514e491e9f02b7f9d92aa6f6 (127.10.130.195). New cstate: current_term: 1 leader_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e1b6db9514e491e9f02b7f9d92aa6f6" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 42703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3de25d8e0bde44a99473e85d85f9be02" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46105 } health_report { overall_health: UNKNOWN } } }
W20260924 07:17:45.266251 27911 tablet.cc:2369] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:17:45.272686 28191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.270586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59648 (local address 127.10.130.195:42703)
0924 07:17:45.270638 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.270640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.270648 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:17:45.270681 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.270682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.270698 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.270755 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:17:45.270857 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.271314 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.272018 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.272227 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.272297 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.272403 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:17:45.272502 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:17:45.272571 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.272585 (+ 14us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.272603 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 07:17:45.274408 27998 raft_consensus.cc:1261] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02 [term 1 FOLLOWER]: Refusing update from remote peer 5e1b6db9514e491e9f02b7f9d92aa6f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:45.274443 27864 raft_consensus.cc:1261] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc [term 1 FOLLOWER]: Refusing update from remote peer 5e1b6db9514e491e9f02b7f9d92aa6f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:45.274781 28202 consensus_queue.cc:1035] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3de25d8e0bde44a99473e85d85f9be02" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 46105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:45.274910 28199 consensus_queue.cc:1035] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37a3f5de9a4f4cf19e73f8ebb802acdc" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 46085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:45.277148 28205 mvcc.cc:205] Tried to move back new op lower bound from 7332799550561554432 to 7332799550417858560. Current Snapshot: MvccSnapshot[applied={T|T < 7332799550561554432}]
I20260924 07:17:45.277360 28206 mvcc.cc:205] Tried to move back new op lower bound from 7332799550561554432 to 7332799550417858560. Current Snapshot: MvccSnapshot[applied={T|T < 7332799550561554432}]
I20260924 07:17:45.277721 28207 mvcc.cc:205] Tried to move back new op lower bound from 7332799550561554432 to 7332799550417858560. Current Snapshot: MvccSnapshot[applied={T|T < 7332799550561554432}]
I20260924 07:17:45.279217 27718 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:45.279312 27718 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:45.280045 27718 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2eb93e524fc8470389d0428c80ebc271: Sending DeleteTablet for 3 replicas of tablet 3f8ac5c12b514a0b899d7afe617558ca
I20260924 07:17:45.280330 28112 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f8ac5c12b514a0b899d7afe617558ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:45 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:59638
I20260924 07:17:45.280351 27844 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f8ac5c12b514a0b899d7afe617558ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:45 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:45232
I20260924 07:17:45.280359 27978 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f8ac5c12b514a0b899d7afe617558ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:45 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:43216
I20260924 07:17:45.280531 28215 tablet_replica.cc:330] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc: stopping tablet replica
I20260924 07:17:45.280632 28215 raft_consensus.cc:2229] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:45.280639 28216 tablet_replica.cc:330] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6: stopping tablet replica
I20260924 07:17:45.280733 28215 raft_consensus.cc:2258] T 3f8ac5c12b514a0b899d7afe617558ca P 37a3f5de9a4f4cf19e73f8ebb802acdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:45.280782 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 27781
I20260924 07:17:45.281319 28216 raft_consensus.cc:2229] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:17:45.281459 28216 raft_consensus.cc:2258] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:45.281894 28216 ts_tablet_manager.cc:1905] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:45.283771 28216 ts_tablet_manager.cc:1918] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:17:45.283883 28216 log.cc:1196] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/3f8ac5c12b514a0b899d7afe617558ca
I20260924 07:17:45.284231 28216 ts_tablet_manager.cc:1939] T 3f8ac5c12b514a0b899d7afe617558ca P 5e1b6db9514e491e9f02b7f9d92aa6f6: Deleting consensus metadata
I20260924 07:17:45.284623 27704 catalog_manager.cc:4841] TS 5e1b6db9514e491e9f02b7f9d92aa6f6 (127.10.130.195:42703): tablet 3f8ac5c12b514a0b899d7afe617558ca (table test-table [id=c7fa9223e67840148c0683ab2dddbf1f]) successfully deleted
W20260924 07:17:45.286847 27703 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:46085 (error 108)
I20260924 07:17:45.286938 28217 tablet_replica.cc:330] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02: stopping tablet replica
I20260924 07:17:45.287038 28217 raft_consensus.cc:2229] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:45.287065 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 27915
I20260924 07:17:45.287147 28217 raft_consensus.cc:2258] T 3f8ac5c12b514a0b899d7afe617558ca P 3de25d8e0bde44a99473e85d85f9be02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:45.291023 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.290896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:46085 (local address 127.0.0.1:45248)
0924 07:17:45.290939 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.290986 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:46085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260924 07:17:45.291060 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 28049
W20260924 07:17:45.292448 27703 catalog_manager.cc:4584] TS 37a3f5de9a4f4cf19e73f8ebb802acdc (127.10.130.193:46085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f8ac5c12b514a0b899d7afe617558ca: Network error: Client connection negotiation failed: client connection to 127.10.130.193:46085: connect: Connection refused (error 111)
I20260924 07:17:45.292500 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.292373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:46105 (local address 127.0.0.1:43222)
0924 07:17:45.292428 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.292454 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.194:46105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 07:17:45.294934 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 27687
2026-09-24T07:17:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1872 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-24T07:17:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:17:45Z Disabled control of system clock
I20260924 07:17:45.319495 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:37737
--webserver_interface=127.10.130.254
--webserver_port=0
--builtin_ntp_servers=127.10.130.212:45877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:37737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 07:17:45.396955 28222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:45.397167 28222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:45.397187 28222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:45.398330 28222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:45.398370 28222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:45.398381 28222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:45.398392 28222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:45.398402 28222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:45.399729 28222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:45877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:37737
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:37737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:45.399951 28222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:45.400246 28222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:45.404593 28228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:45.404582 28230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:45.404582 28227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:45.404919 28222 server_base.cc:1034] running on GCE node
I20260924 07:17:45.405143 28222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:45.405398 28222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:45.406525 28222 hybrid_clock.cc:648] HybridClock initialized: now 1790234265406512 us; error 31 us; skew 500 ppm
I20260924 07:17:45.407598 28222 webserver.cc:460] Webserver started at http://127.10.130.254:34157/ using document root <none> and password file <none>
I20260924 07:17:45.407811 28222 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:45.407856 28222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:45.408010 28222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:45.409005 28222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "03950b15ab8d462d95f2cf89d1622147"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "4fb996300512a9c14424986db3e203e7"
server_key_iv: "9c90bb39e7922c1b172767187a7d97d6"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.409327 28222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "03950b15ab8d462d95f2cf89d1622147"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "4fb996300512a9c14424986db3e203e7"
server_key_iv: "9c90bb39e7922c1b172767187a7d97d6"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.410465 28222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:17:45.411073 28236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.411213 28222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:17:45.411295 28222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "03950b15ab8d462d95f2cf89d1622147"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "4fb996300512a9c14424986db3e203e7"
server_key_iv: "9c90bb39e7922c1b172767187a7d97d6"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.411350 28222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:45.419806 28222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:45.420087 28222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:45.420188 28222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:45.423663 28222 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:37737
I20260924 07:17:45.423691 28288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:37737 every 8 connection(s)
I20260924 07:17:45.424027 28222 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:45.424935 28289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:45.427089 28289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147: Bootstrap starting.
I20260924 07:17:45.427611 28289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:45.427870 28289 log.cc:824] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:45.428462 28289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147: No bootstrap required, opened a new log
I20260924 07:17:45.430035 28289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03950b15ab8d462d95f2cf89d1622147" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37737 } }
I20260924 07:17:45.430166 28289 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:45.430187 28289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03950b15ab8d462d95f2cf89d1622147, State: Initialized, Role: FOLLOWER
I20260924 07:17:45.430320 28289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03950b15ab8d462d95f2cf89d1622147" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37737 } }
I20260924 07:17:45.430380 28289 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:45.430401 28289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:45.430444 28289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:45.430909 28289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03950b15ab8d462d95f2cf89d1622147" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37737 } }
I20260924 07:17:45.430991 28289 leader_election.cc:302] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03950b15ab8d462d95f2cf89d1622147; no voters:
I20260924 07:17:45.431134 28289 leader_election.cc:288] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:17:45.431176 28292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:45.431315 28292 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [term 1 LEADER]: Becoming Leader. State: Replica: 03950b15ab8d462d95f2cf89d1622147, State: Running, Role: LEADER
I20260924 07:17:45.431366 28289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:45.431411 28292 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03950b15ab8d462d95f2cf89d1622147" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37737 } }
I20260924 07:17:45.431756 28294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03950b15ab8d462d95f2cf89d1622147. Latest consensus state: current_term: 1 leader_uuid: "03950b15ab8d462d95f2cf89d1622147" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03950b15ab8d462d95f2cf89d1622147" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37737 } } }
I20260924 07:17:45.431835 28293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03950b15ab8d462d95f2cf89d1622147" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03950b15ab8d462d95f2cf89d1622147" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37737 } } }
I20260924 07:17:45.431901 28293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:45.431838 28294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:45.432493 28301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:45.432956 28301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:45.433380 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 28222
I20260924 07:17:45.433476 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:17:45.433683 10763 external_mini_cluster.cc:1428] Setting key 6593bc1a2f3883eb6e0eb24799c829cd
I20260924 07:17:45.434473 28301 catalog_manager.cc:1347] Generated new cluster ID: e869efb3e56d4bd4b6f438b894e04c19
I20260924 07:17:45.434520 28301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:17:45.438112 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.434120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58182 (local address 127.10.130.254:37737)
0924 07:17:45.434290 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.434298 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.434328 (+ 30us) server_negotiation.cc:407] Connection header received
0924 07:17:45.434433 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.434440 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.434505 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.434648 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:45.434831 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.435680 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.436467 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.436871 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.436976 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.437549 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:45.437553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:45.437553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:45.437714 (+ 161us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:45.437744 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:45.437754 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:45.437806 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.437874 (+ 68us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.437948 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260924 07:17:45.448308 28301 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:17:45.448958 28301 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:45.458366 28301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147: Generated new TSK 0
I20260924 07:17:45.458535 28301 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:17:45.461447 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:37737
--builtin_ntp_servers=127.10.130.212:45877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 07:17:45.539819 28313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:45.540040 28313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:45.540072 28313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:45.541186 28313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:45.541226 28313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:45.541260 28313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:17:45.542577 28313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:45877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:37737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:45.542814 28313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:45.543081 28313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:45.547587 28318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:45.547673 28313 server_base.cc:1034] running on GCE node
W20260924 07:17:45.547577 28319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:45.547847 28321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:45.548069 28313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:45.548352 28313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:45.549515 28313 hybrid_clock.cc:648] HybridClock initialized: now 1790234265549499 us; error 30 us; skew 500 ppm
I20260924 07:17:45.550880 28313 webserver.cc:460] Webserver started at http://127.10.130.193:46829/ using document root <none> and password file <none>
I20260924 07:17:45.551128 28313 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:45.551196 28313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:45.551313 28313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:45.552693 28313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "a3a88be10b9d496da1e395a90baf6b14"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "ca9172373d920265ab4434a158ffaa75"
server_key_iv: "03a24bc399e8fb803f590bd16d4542dc"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.553122 28313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "a3a88be10b9d496da1e395a90baf6b14"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "ca9172373d920265ab4434a158ffaa75"
server_key_iv: "03a24bc399e8fb803f590bd16d4542dc"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.554643 28313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:17:45.555387 28327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.555526 28313 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:45.555606 28313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "a3a88be10b9d496da1e395a90baf6b14"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "ca9172373d920265ab4434a158ffaa75"
server_key_iv: "03a24bc399e8fb803f590bd16d4542dc"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.555696 28313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:45.573987 28313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:45.574236 28313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:45.574330 28313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:45.574494 28313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:45.574754 28313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:45.574781 28313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.574801 28313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:45.574815 28313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.580077 28313 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:38559
I20260924 07:17:45.580142 28440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:38559 every 8 connection(s)
I20260924 07:17:45.580426 28313 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:17:45.585026 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.581052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:55957 (local address 127.10.130.254:37737)
0924 07:17:45.581095 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.581097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.581332 (+ 235us) server_negotiation.cc:407] Connection header received
0924 07:17:45.581789 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.581789 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.581800 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.581847 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:45.582455 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.582932 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.583865 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.584082 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.584128 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.584849 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:45.584849 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:45.584850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:45.584853 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:45.584863 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:45.584866 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:45.584894 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.584952 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.584970 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:17:45.585218 28443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.581039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:37737 (local address 127.10.130.193:55957)
0924 07:17:45.581205 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.581237 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.581714 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.581906 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.581909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.581917 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:45.582394 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.582398 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.583013 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.583014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.583796 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.583798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.584138 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.584138 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.584236 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.584741 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:17:45.584741 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:45.584768 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:17:45.584790 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:45.584900 (+ 110us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:45.584902 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:45.584904 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.584947 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.585020 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260924 07:17:45.585669 28441 heartbeater.cc:347] Connected to a master server at 127.10.130.254:37737
I20260924 07:17:45.585745 28441 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:45.585883 28441 heartbeater.cc:511] Master 127.10.130.254:37737 requested a full tablet report, sending...
I20260924 07:17:45.586194 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 28313
I20260924 07:17:45.586301 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:17:45.586372 28253 ts_manager.cc:194] Registered new tserver with Master: a3a88be10b9d496da1e395a90baf6b14 (127.10.130.193:38559)
I20260924 07:17:45.586514 10763 external_mini_cluster.cc:1428] Setting key e0bb581d17b8284f816e1e8b72d5805f
I20260924 07:17:45.587040 28253 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.130.193:55957
I20260924 07:17:45.587437 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:37737
--builtin_ntp_servers=127.10.130.212:45877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 07:17:45.664544 28444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:45.664739 28444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:45.664767 28444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:45.665866 28444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:45.665902 28444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:45.665933 28444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:17:45.667459 28444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:45877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:37737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:45.667707 28444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:45.667964 28444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 07:17:45.672691 28444 server_base.cc:1034] running on GCE node
W20260924 07:17:45.672627 28449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:45.672626 28452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:45.672631 28450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:45.673017 28444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:45.673236 28444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:45.674368 28444 hybrid_clock.cc:648] HybridClock initialized: now 1790234265674355 us; error 27 us; skew 500 ppm
I20260924 07:17:45.675519 28444 webserver.cc:460] Webserver started at http://127.10.130.194:46879/ using document root <none> and password file <none>
I20260924 07:17:45.675719 28444 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:45.675765 28444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:45.675884 28444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:45.676918 28444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "dfcc0fcdfdca43e6aba61f40cef11c10"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "e327c492465cf6b416d6f79a7d5b7583"
server_key_iv: "7bfce79fc0173d85233c6ff4554970e0"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.677232 28444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "dfcc0fcdfdca43e6aba61f40cef11c10"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "e327c492465cf6b416d6f79a7d5b7583"
server_key_iv: "7bfce79fc0173d85233c6ff4554970e0"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.678356 28444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:17:45.678930 28458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.679077 28444 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:45.679152 28444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "dfcc0fcdfdca43e6aba61f40cef11c10"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "e327c492465cf6b416d6f79a7d5b7583"
server_key_iv: "7bfce79fc0173d85233c6ff4554970e0"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.679211 28444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:45.700555 28444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:45.700814 28444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:45.700927 28444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:45.701128 28444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:45.701448 28444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:45.701489 28444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.701516 28444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:45.701540 28444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.706506 28444 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:44133
I20260924 07:17:45.706579 28571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:44133 every 8 connection(s)
I20260924 07:17:45.706844 28444 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:17:45.711663 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 28444
I20260924 07:17:45.711697 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.707522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:35591 (local address 127.10.130.254:37737)
0924 07:17:45.707572 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.707574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.707784 (+ 210us) server_negotiation.cc:407] Connection header received
0924 07:17:45.708296 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.708298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.708310 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.708359 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:45.708977 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.709466 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.710390 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.710602 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.710657 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.711411 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:45.711413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:45.711413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:45.711418 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:45.711434 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:45.711438 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:45.711508 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.711590 (+ 82us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.711618 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 07:17:45.711772 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:17:45.711876 28574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.707485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:37737 (local address 127.10.130.194:35591)
0924 07:17:45.707674 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.707706 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.708203 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.708427 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.708430 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.708440 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:45.708898 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.708902 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.709542 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.709543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.710315 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.710317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.710669 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.710670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.710797 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.711285 (+ 488us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:17:45.711285 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:45.711312 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:17:45.711334 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:45.711511 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:45.711516 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:45.711519 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.711583 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.711679 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260924 07:17:45.712010 10763 external_mini_cluster.cc:1428] Setting key c90deeb86c76dc9e3cfcddb057715fa9
I20260924 07:17:45.712358 28572 heartbeater.cc:347] Connected to a master server at 127.10.130.254:37737
I20260924 07:17:45.712441 28572 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:45.712620 28572 heartbeater.cc:511] Master 127.10.130.254:37737 requested a full tablet report, sending...
I20260924 07:17:45.712896 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:37737
--builtin_ntp_servers=127.10.130.212:45877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 07:17:45.712996 28253 ts_manager.cc:194] Registered new tserver with Master: dfcc0fcdfdca43e6aba61f40cef11c10 (127.10.130.194:44133)
I20260924 07:17:45.713333 28253 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.130.194:35591
W20260924 07:17:45.790858 28575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:45.791039 28575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:45.791059 28575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:45.792213 28575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:45.792249 28575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:45.792273 28575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:17:45.793552 28575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:45877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:37737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:45.793771 28575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:45.794025 28575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:45.798559 28583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:45.798579 28575 server_base.cc:1034] running on GCE node
W20260924 07:17:45.798564 28580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:45.798631 28581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:45.798926 28575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:45.799149 28575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:45.800282 28575 hybrid_clock.cc:648] HybridClock initialized: now 1790234265800266 us; error 26 us; skew 500 ppm
I20260924 07:17:45.801456 28575 webserver.cc:460] Webserver started at http://127.10.130.195:34753/ using document root <none> and password file <none>
I20260924 07:17:45.801690 28575 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:45.801738 28575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:45.801862 28575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:45.802873 28575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "b736965b1fba4f8593184ffc8d63ad97"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "fd94c52a024d15358c2f0f9dce65d0a8"
server_key_iv: "43a2d1d03a636af11d0931783c2ca22a"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.803194 28575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "b736965b1fba4f8593184ffc8d63ad97"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "fd94c52a024d15358c2f0f9dce65d0a8"
server_key_iv: "43a2d1d03a636af11d0931783c2ca22a"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.804430 28575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:17:45.805074 28589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.805219 28575 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:45.805296 28575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "b736965b1fba4f8593184ffc8d63ad97"
format_stamp: "Formatted at 2026-09-24 07:17:45 on dist-test-slave-7q7m"
server_key: "fd94c52a024d15358c2f0f9dce65d0a8"
server_key_iv: "43a2d1d03a636af11d0931783c2ca22a"
server_key_version: "encryptionkey@0"
I20260924 07:17:45.805359 28575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:45.813803 28575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:45.814056 28575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:45.814174 28575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:45.814359 28575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:45.814670 28575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:45.814703 28575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.814729 28575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:45.814746 28575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:45.819962 28575 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:40281
I20260924 07:17:45.820070 28702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:40281 every 8 connection(s)
I20260924 07:17:45.820361 28575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:17:45.825205 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.820994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:52935 (local address 127.10.130.254:37737)
0924 07:17:45.821035 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.821037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.821258 (+ 221us) server_negotiation.cc:407] Connection header received
0924 07:17:45.821736 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.821737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.821747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.821796 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:45.822425 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.822923 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.823933 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.824213 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.824265 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.825017 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:45.825018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:45.825018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:45.825022 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:45.825032 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:45.825035 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:45.825066 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.825125 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.825145 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:17:45.825420 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.820977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:37737 (local address 127.10.130.195:52935)
0924 07:17:45.821130 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.821163 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.821658 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.821853 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.821856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.821864 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:45.822351 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.822355 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.823020 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.823021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.823858 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.823861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.824276 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.824277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.824384 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.824903 (+ 519us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:17:45.824904 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:45.824933 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:17:45.824955 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:45.825069 (+ 114us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:45.825072 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:45.825075 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.825121 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.825201 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":73,"threads_started":1}
I20260924 07:17:45.825858 28703 heartbeater.cc:347] Connected to a master server at 127.10.130.254:37737
I20260924 07:17:45.825943 28703 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:45.826119 28703 heartbeater.cc:511] Master 127.10.130.254:37737 requested a full tablet report, sending...
I20260924 07:17:45.826442 28253 ts_manager.cc:194] Registered new tserver with Master: b736965b1fba4f8593184ffc8d63ad97 (127.10.130.195:40281)
I20260924 07:17:45.826803 28253 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.130.195:52935
I20260924 07:17:45.827462 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 28575
I20260924 07:17:45.827550 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:17:45.827739 10763 external_mini_cluster.cc:1428] Setting key d7beef0028673f1fa60525b7e44ffa82
I20260924 07:17:45.828500 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:17:45.832522 28312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.829771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58186 (local address 127.10.130.254:37737)
0924 07:17:45.829819 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.829820 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.829925 (+ 105us) server_negotiation.cc:407] Connection header received
0924 07:17:45.829984 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.829985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.829998 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.830058 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:45.830207 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.830681 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.831459 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.831657 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.831712 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.832345 (+ 633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:45.832347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:45.832347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:45.832351 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:45.832364 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:45.832368 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:45.832407 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.832439 (+ 32us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.832461 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 07:17:45.833859 28253 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:17:45.835377 28253 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:17:45.841454 28713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.837546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52784 (local address 127.10.130.193:38559)
0924 07:17:45.837884 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.837889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.837897 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:17:45.837934 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.837937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.837951 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.838053 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:45.838264 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.839404 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.840568 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.841060 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.841132 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.841226 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.841325 (+ 99us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.841363 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":83,"threads_started":1}
I20260924 07:17:45.841444 28712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.837446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38559 (local address 127.0.0.1:52784)
0924 07:17:45.837641 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.837681 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.837747 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.838057 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.838058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.838060 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:45.838174 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.838175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.839494 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.839495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.840499 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.840501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.841146 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.841147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.841223 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.841236 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.841312 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.841364 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":73,"thread_start_us":70,"threads_started":1}
I20260924 07:17:45.841784 28716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.837879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.10.130.195:40281)
0924 07:17:45.838054 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.838057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.838066 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:17:45.838098 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.838099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.838112 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.838176 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:45.838351 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.839058 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.840407 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.841154 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.841221 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.841320 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.841655 (+ 335us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.841699 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":75,"threads_started":1}
I20260924 07:17:45.841969 28715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.837684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:40281 (local address 127.0.0.1:54380)
0924 07:17:45.837891 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.837902 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.837944 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.838234 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.838235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.838237 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:45.838281 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.838283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.839134 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.839135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.840323 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.840326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.841352 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.841353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.841419 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.841424 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.841473 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.841903 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":60,"threads_started":1}
I20260924 07:17:45.842273 28714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.837559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58308 (local address 127.10.130.194:44133)
0924 07:17:45.837784 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.837789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.837801 (+ 12us) server_negotiation.cc:407] Connection header received
0924 07:17:45.837847 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.837849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.837866 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.837960 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:45.838238 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.839404 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.841243 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.841936 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.842010 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.842108 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.842157 (+ 49us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.842194 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":95,"threads_started":1}
I20260924 07:17:45.842247 28375 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3940c064d241c2938e5e7ca7c0824a (DEFAULT_TABLE table=test-table [id=a1f034479e644fe5a04be637119899ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:45.842504 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.837448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:44133 (local address 127.0.0.1:58308)
0924 07:17:45.837708 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.837717 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.837755 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.838055 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.838058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.838063 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:45.838147 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.838149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.840082 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.840083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.841145 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.841148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.842020 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.842021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.842081 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.842085 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.842143 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.842438 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":1,"spinlock_wait_cycles":896,"thread_start_us":118,"threads_started":1}
I20260924 07:17:45.842609 28375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3940c064d241c2938e5e7ca7c0824a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:45.842803 28637 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3940c064d241c2938e5e7ca7c0824a (DEFAULT_TABLE table=test-table [id=a1f034479e644fe5a04be637119899ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:45.843079 28637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3940c064d241c2938e5e7ca7c0824a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:45.844682 28506 tablet_service.cc:1467] Processing CreateTablet for tablet 8f3940c064d241c2938e5e7ca7c0824a (DEFAULT_TABLE table=test-table [id=a1f034479e644fe5a04be637119899ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:45.845000 28506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f3940c064d241c2938e5e7ca7c0824a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:45.845525 28717 tablet_bootstrap.cc:492] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14: Bootstrap starting.
I20260924 07:17:45.845990 28718 tablet_bootstrap.cc:492] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97: Bootstrap starting.
I20260924 07:17:45.846242 28717 tablet_bootstrap.cc:654] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:45.846537 28717 log.cc:824] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:45.846575 28718 tablet_bootstrap.cc:654] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:45.846850 28718 log.cc:824] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:45.847261 28717 tablet_bootstrap.cc:492] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14: No bootstrap required, opened a new log
I20260924 07:17:45.847314 28717 ts_tablet_manager.cc:1397] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:17:45.847689 28721 tablet_bootstrap.cc:492] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10: Bootstrap starting.
I20260924 07:17:45.848222 28721 tablet_bootstrap.cc:654] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:45.848490 28721 log.cc:824] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:45.849133 28721 tablet_bootstrap.cc:492] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10: No bootstrap required, opened a new log
I20260924 07:17:45.849186 28721 ts_tablet_manager.cc:1397] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:17:45.849275 28717 raft_consensus.cc:348] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a88be10b9d496da1e395a90baf6b14" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38559 } } peers { permanent_uuid: "dfcc0fcdfdca43e6aba61f40cef11c10" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44133 } } peers { permanent_uuid: "b736965b1fba4f8593184ffc8d63ad97" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40281 } }
I20260924 07:17:45.849401 28717 raft_consensus.cc:374] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:45.849421 28717 raft_consensus.cc:729] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3a88be10b9d496da1e395a90baf6b14, State: Initialized, Role: FOLLOWER
I20260924 07:17:45.849542 28717 consensus_queue.cc:260] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a88be10b9d496da1e395a90baf6b14" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38559 } } peers { permanent_uuid: "dfcc0fcdfdca43e6aba61f40cef11c10" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44133 } } peers { permanent_uuid: "b736965b1fba4f8593184ffc8d63ad97" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40281 } }
I20260924 07:17:45.849787 28717 ts_tablet_manager.cc:1428] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260924 07:17:45.849843 28441 heartbeater.cc:503] Master 127.10.130.254:37737 was elected leader, sending a full tablet report...
I20260924 07:17:45.850332 28718 tablet_bootstrap.cc:492] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97: No bootstrap required, opened a new log
I20260924 07:17:45.850399 28718 ts_tablet_manager.cc:1397] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260924 07:17:45.851282 28721 raft_consensus.cc:348] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a88be10b9d496da1e395a90baf6b14" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38559 } } peers { permanent_uuid: "dfcc0fcdfdca43e6aba61f40cef11c10" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44133 } } peers { permanent_uuid: "b736965b1fba4f8593184ffc8d63ad97" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40281 } }
I20260924 07:17:45.851398 28721 raft_consensus.cc:374] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:45.851428 28721 raft_consensus.cc:729] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfcc0fcdfdca43e6aba61f40cef11c10, State: Initialized, Role: FOLLOWER
I20260924 07:17:45.851533 28721 consensus_queue.cc:260] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a88be10b9d496da1e395a90baf6b14" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38559 } } peers { permanent_uuid: "dfcc0fcdfdca43e6aba61f40cef11c10" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44133 } } peers { permanent_uuid: "b736965b1fba4f8593184ffc8d63ad97" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40281 } }
I20260924 07:17:45.851758 28721 ts_tablet_manager.cc:1428] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:17:45.851809 28572 heartbeater.cc:503] Master 127.10.130.254:37737 was elected leader, sending a full tablet report...
I20260924 07:17:45.852566 28718 raft_consensus.cc:348] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a88be10b9d496da1e395a90baf6b14" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38559 } } peers { permanent_uuid: "dfcc0fcdfdca43e6aba61f40cef11c10" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44133 } } peers { permanent_uuid: "b736965b1fba4f8593184ffc8d63ad97" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40281 } }
I20260924 07:17:45.852717 28718 raft_consensus.cc:374] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:45.852753 28718 raft_consensus.cc:729] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b736965b1fba4f8593184ffc8d63ad97, State: Initialized, Role: FOLLOWER
I20260924 07:17:45.852860 28718 consensus_queue.cc:260] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a88be10b9d496da1e395a90baf6b14" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38559 } } peers { permanent_uuid: "dfcc0fcdfdca43e6aba61f40cef11c10" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44133 } } peers { permanent_uuid: "b736965b1fba4f8593184ffc8d63ad97" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40281 } }
I20260924 07:17:45.853045 28718 ts_tablet_manager.cc:1428] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:17:45.853121 28703 heartbeater.cc:503] Master 127.10.130.254:37737 was elected leader, sending a full tablet report...
I20260924 07:17:45.921316 28725 raft_consensus.cc:482] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:17:45.921443 28725 raft_consensus.cc:504] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a88be10b9d496da1e395a90baf6b14" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38559 } } peers { permanent_uuid: "dfcc0fcdfdca43e6aba61f40cef11c10" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44133 } } peers { permanent_uuid: "b736965b1fba4f8593184ffc8d63ad97" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40281 } }
I20260924 07:17:45.921799 28725 leader_election.cc:288] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3a88be10b9d496da1e395a90baf6b14 (127.10.130.193:38559), dfcc0fcdfdca43e6aba61f40cef11c10 (127.10.130.194:44133)
I20260924 07:17:45.925472 28713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.922028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:50279 (local address 127.10.130.193:38559)
0924 07:17:45.922083 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.922085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.922094 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:17:45.922141 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.922142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.922155 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.922208 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:45.922374 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.923129 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.924594 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.925131 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.925200 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.925206 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.925362 (+ 156us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.925400 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 07:17:45.925472 28714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.922134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:39589 (local address 127.10.130.194:44133)
0924 07:17:45.922189 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.922191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.922203 (+ 12us) server_negotiation.cc:407] Connection header received
0924 07:17:45.922283 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.922283 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.922290 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.922328 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:45.922446 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.923131 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.924576 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.925138 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.925198 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.925206 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.925362 (+ 156us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.925400 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 07:17:45.925478 28705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.921944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38559 (local address 127.10.130.195:50279)
0924 07:17:45.922004 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.922016 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.922061 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.922231 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.922232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.922233 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:45.922305 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.922306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.923195 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.923195 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.924495 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.924499 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.925205 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.925206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.925298 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.925302 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.925350 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.925390 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":33}
I20260924 07:17:45.925487 28726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.921996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:44133 (local address 127.10.130.195:39589)
0924 07:17:45.922157 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.922166 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:45.922217 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:45.922343 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:45.922344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:45.922345 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:45.922395 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.922396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.923194 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.923195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.924495 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:45.924499 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.925206 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.925206 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:45.925302 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.925304 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:17:45.925357 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 07:17:45.925388 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":37,"thread_start_us":93,"threads_started":1}
I20260924 07:17:45.925863 28395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3940c064d241c2938e5e7ca7c0824a" candidate_uuid: "b736965b1fba4f8593184ffc8d63ad97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a88be10b9d496da1e395a90baf6b14" is_pre_election: true
I20260924 07:17:45.925860 28526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3940c064d241c2938e5e7ca7c0824a" candidate_uuid: "b736965b1fba4f8593184ffc8d63ad97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfcc0fcdfdca43e6aba61f40cef11c10" is_pre_election: true
I20260924 07:17:45.926005 28395 raft_consensus.cc:2454] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b736965b1fba4f8593184ffc8d63ad97 in term 0.
I20260924 07:17:45.926005 28526 raft_consensus.cc:2454] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b736965b1fba4f8593184ffc8d63ad97 in term 0.
I20260924 07:17:45.926209 28591 leader_election.cc:302] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b736965b1fba4f8593184ffc8d63ad97, dfcc0fcdfdca43e6aba61f40cef11c10; no voters:
I20260924 07:17:45.926332 28725 raft_consensus.cc:2781] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:17:45.926383 28725 raft_consensus.cc:482] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:17:45.926410 28725 raft_consensus.cc:3037] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:45.927062 28725 raft_consensus.cc:504] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a88be10b9d496da1e395a90baf6b14" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38559 } } peers { permanent_uuid: "dfcc0fcdfdca43e6aba61f40cef11c10" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44133 } } peers { permanent_uuid: "b736965b1fba4f8593184ffc8d63ad97" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40281 } }
I20260924 07:17:45.927172 28725 leader_election.cc:288] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [CANDIDATE]: Term 1 election: Requested vote from peers a3a88be10b9d496da1e395a90baf6b14 (127.10.130.193:38559), dfcc0fcdfdca43e6aba61f40cef11c10 (127.10.130.194:44133)
I20260924 07:17:45.927309 28395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3940c064d241c2938e5e7ca7c0824a" candidate_uuid: "b736965b1fba4f8593184ffc8d63ad97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3a88be10b9d496da1e395a90baf6b14"
I20260924 07:17:45.927348 28526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f3940c064d241c2938e5e7ca7c0824a" candidate_uuid: "b736965b1fba4f8593184ffc8d63ad97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfcc0fcdfdca43e6aba61f40cef11c10"
I20260924 07:17:45.927379 28395 raft_consensus.cc:3037] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:45.927426 28526 raft_consensus.cc:3037] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:45.928020 28395 raft_consensus.cc:2454] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b736965b1fba4f8593184ffc8d63ad97 in term 1.
I20260924 07:17:45.928182 28592 leader_election.cc:302] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3a88be10b9d496da1e395a90baf6b14, b736965b1fba4f8593184ffc8d63ad97; no voters:
I20260924 07:17:45.928188 28526 raft_consensus.cc:2454] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b736965b1fba4f8593184ffc8d63ad97 in term 1.
I20260924 07:17:45.928296 28725 raft_consensus.cc:2781] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:45.928452 28725 raft_consensus.cc:686] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [term 1 LEADER]: Becoming Leader. State: Replica: b736965b1fba4f8593184ffc8d63ad97, State: Running, Role: LEADER
I20260924 07:17:45.928632 28725 consensus_queue.cc:237] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a88be10b9d496da1e395a90baf6b14" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38559 } } peers { permanent_uuid: "dfcc0fcdfdca43e6aba61f40cef11c10" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44133 } } peers { permanent_uuid: "b736965b1fba4f8593184ffc8d63ad97" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40281 } }
I20260924 07:17:45.929411 28253 catalog_manager.cc:5495] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 reported cstate change: term changed from 0 to 1, leader changed from <none> to b736965b1fba4f8593184ffc8d63ad97 (127.10.130.195). New cstate: current_term: 1 leader_uuid: "b736965b1fba4f8593184ffc8d63ad97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3a88be10b9d496da1e395a90baf6b14" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfcc0fcdfdca43e6aba61f40cef11c10" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b736965b1fba4f8593184ffc8d63ad97" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40281 } health_report { overall_health: HEALTHY } } }
I20260924 07:17:45.934857 28716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.932775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54386 (local address 127.10.130.195:40281)
0924 07:17:45.932816 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:45.932818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:45.932826 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:17:45.932888 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:45.932889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:45.932905 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:45.932962 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:17:45.933056 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.933491 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.934191 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:45.934373 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:45.934424 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:45.934558 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:17:45.934657 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:17:45.934724 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:45.934759 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 07:17:45.934777 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 07:17:45.936437 28395 raft_consensus.cc:1261] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14 [term 1 FOLLOWER]: Refusing update from remote peer b736965b1fba4f8593184ffc8d63ad97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:45.936480 28526 raft_consensus.cc:1261] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10 [term 1 FOLLOWER]: Refusing update from remote peer b736965b1fba4f8593184ffc8d63ad97: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:45.936738 28727 consensus_queue.cc:1035] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3a88be10b9d496da1e395a90baf6b14" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:45.936854 28725 consensus_queue.cc:1035] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfcc0fcdfdca43e6aba61f40cef11c10" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 44133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:45.938720 28730 mvcc.cc:205] Tried to move back new op lower bound from 7332799553273413632 to 7332799553244377088. Current Snapshot: MvccSnapshot[applied={T|T < 7332799553273413632}]
I20260924 07:17:45.939569 28732 mvcc.cc:205] Tried to move back new op lower bound from 7332799553273413632 to 7332799553244377088. Current Snapshot: MvccSnapshot[applied={T|T < 7332799553273413632}]
I20260924 07:17:45.939702 28731 mvcc.cc:205] Tried to move back new op lower bound from 7332799553273413632 to 7332799553244377088. Current Snapshot: MvccSnapshot[applied={T|T < 7332799553273413632 or (T in {7332799553273413632})}]
I20260924 07:17:45.941033 28252 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:45.941108 28252 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:45.941776 28252 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03950b15ab8d462d95f2cf89d1622147: Sending DeleteTablet for 3 replicas of tablet 8f3940c064d241c2938e5e7ca7c0824a
I20260924 07:17:45.942050 28506 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f3940c064d241c2938e5e7ca7c0824a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:45 UTC) from {username='slave'} at 127.0.0.1:58308
I20260924 07:17:45.942093 28375 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f3940c064d241c2938e5e7ca7c0824a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:45 UTC) from {username='slave'} at 127.0.0.1:52784
I20260924 07:17:45.942088 28637 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f3940c064d241c2938e5e7ca7c0824a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:45 UTC) from {username='slave'} at 127.0.0.1:54380
I20260924 07:17:45.942322 28739 tablet_replica.cc:330] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14: stopping tablet replica
I20260924 07:17:45.942411 28739 raft_consensus.cc:2229] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:45.942505 28739 raft_consensus.cc:2258] T 8f3940c064d241c2938e5e7ca7c0824a P a3a88be10b9d496da1e395a90baf6b14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:45.942569 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 28313
I20260924 07:17:45.942652 28740 tablet_replica.cc:330] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10: stopping tablet replica
I20260924 07:17:45.942728 28740 raft_consensus.cc:2229] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:45.942797 28740 raft_consensus.cc:2258] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:45.943131 28741 tablet_replica.cc:330] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97: stopping tablet replica
I20260924 07:17:45.943214 28741 raft_consensus.cc:2229] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:17:45.943312 28741 raft_consensus.cc:2258] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:45.943637 28741 ts_tablet_manager.cc:1905] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:45.944278 28740 ts_tablet_manager.cc:1905] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:45.945148 28741 ts_tablet_manager.cc:1918] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:17:45.945218 28741 log.cc:1196] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/8f3940c064d241c2938e5e7ca7c0824a
I20260924 07:17:45.945447 28741 ts_tablet_manager.cc:1939] T 8f3940c064d241c2938e5e7ca7c0824a P b736965b1fba4f8593184ffc8d63ad97: Deleting consensus metadata
I20260924 07:17:45.945955 28239 catalog_manager.cc:4841] TS b736965b1fba4f8593184ffc8d63ad97 (127.10.130.195:40281): tablet 8f3940c064d241c2938e5e7ca7c0824a (table test-table [id=a1f034479e644fe5a04be637119899ed]) successfully deleted
I20260924 07:17:45.946170 28740 ts_tablet_manager.cc:1918] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:17:45.946228 28740 log.cc:1196] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/8f3940c064d241c2938e5e7ca7c0824a
I20260924 07:17:45.946416 28740 ts_tablet_manager.cc:1939] T 8f3940c064d241c2938e5e7ca7c0824a P dfcc0fcdfdca43e6aba61f40cef11c10: Deleting consensus metadata
I20260924 07:17:45.946847 28238 catalog_manager.cc:4841] TS dfcc0fcdfdca43e6aba61f40cef11c10 (127.10.130.194:44133): tablet 8f3940c064d241c2938e5e7ca7c0824a (table test-table [id=a1f034479e644fe5a04be637119899ed]) successfully deleted
W20260924 07:17:45.949445 28239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:38559 (error 108)
I20260924 07:17:45.949648 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 28444
I20260924 07:17:45.950220 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:45.949770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38559 (local address 127.0.0.1:52800)
0924 07:17:45.950147 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:45.950187 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:38559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357}
W20260924 07:17:45.950340 28239 catalog_manager.cc:4584] TS a3a88be10b9d496da1e395a90baf6b14 (127.10.130.193:38559): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f3940c064d241c2938e5e7ca7c0824a: Network error: Client connection negotiation failed: client connection to 127.10.130.193:38559: connect: Connection refused (error 111)
I20260924 07:17:45.953727 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 28575
I20260924 07:17:45.958057 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 28222
2026-09-24T07:17:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (662 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:17:46 dist-test-slave-7q7m krb5kdc[28745](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:17:46 dist-test-slave-7q7m krb5kdc[28745](info): set up 2 sockets
Sep 24 07:17:46 dist-test-slave-7q7m krb5kdc[28745](info): commencing operation
krb5kdc: starting...
W20260924 07:17:46.899433 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:17:46 dist-test-slave-7q7m krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:17:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:17:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:17:47.008145 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:43627
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:43627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:47.088853 28758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:47.089046 28758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:47.089074 28758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:47.090173 28758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:47.090211 28758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:47.090229 28758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:47.090245 28758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:47.090258 28758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:47.091667 28758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:43627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:43627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:47.091895 28758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:47.092118 28758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:47.096486 28764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:47.096495 28766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:47.096690 28758 server_base.cc:1034] running on GCE node
W20260924 07:17:47.096495 28763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:47.096951 28758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:47.097223 28758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:47.098358 28758 hybrid_clock.cc:648] HybridClock initialized: now 1790234267098345 us; error 27 us; skew 500 ppm
Sep 24 07:17:47 dist-test-slave-7q7m krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234267, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:47.100701 28758 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:17:47.101061 28758 webserver.cc:460] Webserver started at http://127.10.130.254:42127/ using document root <none> and password file <none>
I20260924 07:17:47.101245 28758 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:47.101282 28758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:47.101383 28758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:47.102373 28758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "70512f55626a4c158d2d7c74fac337b3"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "0599f80395676e3d45ad3842b87ee3f7"
server_key_iv: "cc9342f8d9c3d3dd50db39dc16a2fe7c"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.102654 28758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "70512f55626a4c158d2d7c74fac337b3"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "0599f80395676e3d45ad3842b87ee3f7"
server_key_iv: "cc9342f8d9c3d3dd50db39dc16a2fe7c"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.103780 28758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:17:47.104516 28773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:47.104712 28758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 07:17:47.104796 28758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "70512f55626a4c158d2d7c74fac337b3"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "0599f80395676e3d45ad3842b87ee3f7"
server_key_iv: "cc9342f8d9c3d3dd50db39dc16a2fe7c"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.104857 28758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:47.126824 28758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:47.128990 28758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:47.129141 28758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:47.132843 28758 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:43627
I20260924 07:17:47.132881 28825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:43627 every 8 connection(s)
I20260924 07:17:47.133208 28758 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:47.134079 28826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:47.136404 28826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3: Bootstrap starting.
I20260924 07:17:47.136956 28826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:47.137207 28826 log.cc:824] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:47.137904 28826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3: No bootstrap required, opened a new log
I20260924 07:17:47.139415 28826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70512f55626a4c158d2d7c74fac337b3" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43627 } }
I20260924 07:17:47.139549 28826 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:47.139572 28826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70512f55626a4c158d2d7c74fac337b3, State: Initialized, Role: FOLLOWER
I20260924 07:17:47.139721 28826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70512f55626a4c158d2d7c74fac337b3" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43627 } }
I20260924 07:17:47.139793 28826 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:47.139820 28826 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:47.139864 28826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:47.140411 28826 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70512f55626a4c158d2d7c74fac337b3" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43627 } }
I20260924 07:17:47.140498 28826 leader_election.cc:302] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70512f55626a4c158d2d7c74fac337b3; no voters:
I20260924 07:17:47.140663 28826 leader_election.cc:288] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:17:47.140702 28829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:47.140843 28829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [term 1 LEADER]: Becoming Leader. State: Replica: 70512f55626a4c158d2d7c74fac337b3, State: Running, Role: LEADER
I20260924 07:17:47.140892 28826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:47.140928 28829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70512f55626a4c158d2d7c74fac337b3" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43627 } }
I20260924 07:17:47.141446 28831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70512f55626a4c158d2d7c74fac337b3. Latest consensus state: current_term: 1 leader_uuid: "70512f55626a4c158d2d7c74fac337b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70512f55626a4c158d2d7c74fac337b3" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43627 } } }
I20260924 07:17:47.141526 28831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:47.141822 28830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70512f55626a4c158d2d7c74fac337b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70512f55626a4c158d2d7c74fac337b3" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43627 } } }
I20260924 07:17:47.141891 28838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:47.141896 28830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:47.142377 28838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:47.142563 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 28758
I20260924 07:17:47.142688 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:17:47.142904 10763 external_mini_cluster.cc:1428] Setting key 2fb3d229bf4d44176f8712689254c9dd
I20260924 07:17:47.143886 28838 catalog_manager.cc:1347] Generated new cluster ID: a6859ce87c7246759fae1def1fa93608
I20260924 07:17:47.143934 28838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:17:47 dist-test-slave-7q7m krb5kdc[28745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:47.154762 28848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.143391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32954 (local address 127.10.130.254:43627)
0924 07:17:47.144346 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:47.144354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:47.144388 (+ 34us) server_negotiation.cc:407] Connection header received
0924 07:17:47.144497 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:47.144505 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:47.144560 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:47.144698 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:47.145362 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.146778 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.147779 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.148427 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.148538 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.151256 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:47.151261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:47.151262 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:47.151275 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:47.153181 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:47.153563 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:47.153566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:47.153566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:47.153685 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:47.153864 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:47.153865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:47.153865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:47.154169 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:47.154292 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:47.154400 (+ 108us) server_negotiation.cc:299] Negotiation successful
0924 07:17:47.154528 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":914,"thread_start_us":74,"threads_started":1}
I20260924 07:17:47.163059 28838 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:17:47.163544 28838 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:47.169783 28838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3: Generated new TSK 0
I20260924 07:17:47.169973 28838 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:17:47.197759 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43627
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:47.275305 28852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:47.275487 28852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:47.275508 28852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:47.276636 28852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:47.276674 28852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:47.276701 28852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:17:47.278085 28852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28852
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:47.278287 28852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:47.278489 28852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 07:17:47.283136 28852 server_base.cc:1034] running on GCE node
W20260924 07:17:47.283174 28857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:47.283099 28860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:47.283107 28858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:47.283428 28852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:47.283648 28852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:47.284780 28852 hybrid_clock.cc:648] HybridClock initialized: now 1790234267284767 us; error 28 us; skew 500 ppm
Sep 24 07:17:47 dist-test-slave-7q7m krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234267, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:47.286955 28852 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:17:47.287330 28852 webserver.cc:460] Webserver started at http://127.10.130.193:33939/ using document root <none> and password file <none>
I20260924 07:17:47.287524 28852 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:47.287564 28852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:47.287685 28852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:47.288679 28852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "04108a981f994d909a017d8b146015eb"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "e7a1ceb11a40e0f0d62d4620c7694936"
server_key_iv: "c380e9b26d30949b69db7b009784d1fb"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.288980 28852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "04108a981f994d909a017d8b146015eb"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "e7a1ceb11a40e0f0d62d4620c7694936"
server_key_iv: "c380e9b26d30949b69db7b009784d1fb"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.290109 28852 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:17:47.290700 28867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:47.290825 28852 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:17:47.290898 28852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "04108a981f994d909a017d8b146015eb"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "e7a1ceb11a40e0f0d62d4620c7694936"
server_key_iv: "c380e9b26d30949b69db7b009784d1fb"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.290952 28852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:47.317030 28852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:47.318766 28852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:47.318897 28852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:47.319113 28852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:47.319407 28852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:47.319439 28852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:47.319468 28852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:47.319489 28852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:47.324811 28852 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:35985
I20260924 07:17:47.324869 28980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:35985 every 8 connection(s)
I20260924 07:17:47.325160 28852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
Sep 24 07:17:47 dist-test-slave-7q7m krb5kdc[28745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234267, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:47.332413 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 28852
I20260924 07:17:47.332547 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:17:47.332815 10763 external_mini_cluster.cc:1428] Setting key cd8be49b306acadafc076c0aed43631c
I20260924 07:17:47.334424 28848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.325845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:46953 (local address 127.10.130.254:43627)
0924 07:17:47.325912 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:47.325915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:47.326115 (+ 200us) server_negotiation.cc:407] Connection header received
0924 07:17:47.326621 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:47.326622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:47.326636 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:47.326676 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:47.327687 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.328264 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.329284 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.329487 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.329534 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.331561 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:47.331563 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:47.331563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:47.331570 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:47.333075 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:47.333506 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:47.333507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:47.333507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:47.333585 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:47.333807 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:47.333807 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:47.333808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:47.333939 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:47.334012 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:47.334235 (+ 223us) server_negotiation.cc:299] Negotiation successful
0924 07:17:47.334327 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 07:17:47.334494 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.325801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:43627 (local address 127.10.130.193:46953)
0924 07:17:47.325980 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:47.326020 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:47.326509 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:47.326755 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:47.326759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:47.327389 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:47.327594 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.327599 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.328366 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.328367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.329205 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.329207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.329545 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.329546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.329649 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.330151 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:47.330159 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:47.331459 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:47.333152 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:47.333158 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:47.333161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:47.333438 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:47.333648 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:47.333649 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:47.333649 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:47.333750 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:47.333998 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:47.334001 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:47.334122 (+ 121us) client_negotiation.cc:769] Sending connection context
0924 07:17:47.334198 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 07:17:47.334321 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260924 07:17:47.335474 28981 heartbeater.cc:347] Connected to a master server at 127.10.130.254:43627
I20260924 07:17:47.335566 28981 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:47.335749 28981 heartbeater.cc:511] Master 127.10.130.254:43627 requested a full tablet report, sending...
I20260924 07:17:47.336302 28790 ts_manager.cc:194] Registered new tserver with Master: 04108a981f994d909a017d8b146015eb (127.10.130.193:35985)
I20260924 07:17:47.336882 28790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:46953
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:17:47.355906 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43627
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:47.433821 28986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:47.433998 28986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:47.434019 28986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:47.435103 28986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:47.435134 28986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:47.435158 28986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:17:47.436540 28986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:47.436725 28986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:47.436892 28986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:47.441423 28992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:47.441527 28994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:47.441430 28991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:47.441676 28986 server_base.cc:1034] running on GCE node
I20260924 07:17:47.441835 28986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:47.442080 28986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:47.443214 28986 hybrid_clock.cc:648] HybridClock initialized: now 1790234267443201 us; error 29 us; skew 500 ppm
Sep 24 07:17:47 dist-test-slave-7q7m krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234267, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:47.445371 28986 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:17:47.445791 28986 webserver.cc:460] Webserver started at http://127.10.130.194:34399/ using document root <none> and password file <none>
I20260924 07:17:47.445986 28986 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:47.446029 28986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:47.446147 28986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:47.447094 28986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "8e3045ed6e8041eaa357b35cc8006e31"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "6a69bc733a46925d40a7189141e92d3d"
server_key_iv: "2366bcfa7a4244914d3a92fa9475d62b"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.447398 28986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "8e3045ed6e8041eaa357b35cc8006e31"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "6a69bc733a46925d40a7189141e92d3d"
server_key_iv: "2366bcfa7a4244914d3a92fa9475d62b"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.448616 28986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:17:47.449199 29001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:47.449339 28986 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:47.449414 28986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "8e3045ed6e8041eaa357b35cc8006e31"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "6a69bc733a46925d40a7189141e92d3d"
server_key_iv: "2366bcfa7a4244914d3a92fa9475d62b"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.449470 28986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:47.458870 28986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:47.460633 28986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:47.460752 28986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:47.460927 28986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:47.461189 28986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:47.461221 28986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:47.461247 28986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:47.461272 28986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:47.466521 28986 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:38831
I20260924 07:17:47.466558 29114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:38831 every 8 connection(s)
I20260924 07:17:47.466831 28986 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:17:47.470122 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 28986
I20260924 07:17:47.470218 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:17:47.470388 10763 external_mini_cluster.cc:1428] Setting key 40439659106cb8776a8d32bb6bc30717
Sep 24 07:17:47 dist-test-slave-7q7m krb5kdc[28745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234267, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:47.475198 28848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.467395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:33787 (local address 127.10.130.254:43627)
0924 07:17:47.467459 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:47.467462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:47.467640 (+ 178us) server_negotiation.cc:407] Connection header received
0924 07:17:47.468102 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:47.468104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:47.468113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:47.468161 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:47.468968 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.469453 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.470403 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.470658 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.470716 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.472828 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:47.472830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:47.472831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:47.472839 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:47.474021 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:47.474366 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:47.474368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:47.474368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:47.474423 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:47.474630 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:47.474631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:47.474631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:47.474738 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:47.474791 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:47.475029 (+ 238us) server_negotiation.cc:299] Negotiation successful
0924 07:17:47.475108 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 07:17:47.475378 29117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.467369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:43627 (local address 127.10.130.194:33787)
0924 07:17:47.467527 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:47.467559 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:47.468019 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:47.468212 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:47.468215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:47.468724 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:47.468899 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.468903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.469512 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.469512 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.470327 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.470329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.470741 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.470742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.470853 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.471296 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:47.471304 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:47.472747 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:47.474101 (+ 1354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:47.474107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:47.474109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:47.474304 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:47.474473 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:47.474474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:47.474475 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:47.474577 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:47.474790 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:47.474796 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:47.474920 (+ 124us) client_negotiation.cc:769] Sending connection context
0924 07:17:47.475003 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 07:17:47.475181 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260924 07:17:47.475780 29115 heartbeater.cc:347] Connected to a master server at 127.10.130.254:43627
I20260924 07:17:47.475862 29115 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:47.476043 29115 heartbeater.cc:511] Master 127.10.130.254:43627 requested a full tablet report, sending...
I20260924 07:17:47.476404 28790 ts_manager.cc:194] Registered new tserver with Master: 8e3045ed6e8041eaa357b35cc8006e31 (127.10.130.194:38831)
I20260924 07:17:47.476902 28790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:33787
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:17:47.492537 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43627
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:47.571785 29120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:47.572005 29120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:47.572037 29120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:47.573158 29120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:47.573191 29120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:47.573215 29120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:17:47.574558 29120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:47.574743 29120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:47.574903 29120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:47.579523 29126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:47.579532 29125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:47.579533 29128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:47.579586 29120 server_base.cc:1034] running on GCE node
I20260924 07:17:47.579833 29120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:47.580084 29120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:47.581212 29120 hybrid_clock.cc:648] HybridClock initialized: now 1790234267581202 us; error 25 us; skew 500 ppm
Sep 24 07:17:47 dist-test-slave-7q7m krb5kdc[28745](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234267, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:47.583487 29120 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:17:47.583887 29120 webserver.cc:460] Webserver started at http://127.10.130.195:45711/ using document root <none> and password file <none>
I20260924 07:17:47.584127 29120 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:47.584172 29120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:47.584288 29120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:47.585269 29120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "021f3e552d3d4618ada6248999c50a6a"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "ef80e8a5276d6540d1803c3a53ae8321"
server_key_iv: "82c34c418d06c15a292717d0760c711b"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.585577 29120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "021f3e552d3d4618ada6248999c50a6a"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "ef80e8a5276d6540d1803c3a53ae8321"
server_key_iv: "82c34c418d06c15a292717d0760c711b"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.586695 29120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 07:17:47.587312 29135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:47.587455 29120 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:47.587527 29120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "021f3e552d3d4618ada6248999c50a6a"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "ef80e8a5276d6540d1803c3a53ae8321"
server_key_iv: "82c34c418d06c15a292717d0760c711b"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.587589 29120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:47.598028 29120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:47.600036 29120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:47.600169 29120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:47.600351 29120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:47.600627 29120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:47.600660 29120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:47.600688 29120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:47.600711 29120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:47.606045 29120 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:40599
I20260924 07:17:47.606096 29248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:40599 every 8 connection(s)
I20260924 07:17:47.606374 29120 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:17:47.606608 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 29120
I20260924 07:17:47.606721 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:17:47.606920 10763 external_mini_cluster.cc:1428] Setting key c5aac28f0d474f6afbaa16107984a90b
Sep 24 07:17:47 dist-test-slave-7q7m krb5kdc[28745](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234267, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:47.614853 28848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.607069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:40405 (local address 127.10.130.254:43627)
0924 07:17:47.607180 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:47.607182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:47.607408 (+ 226us) server_negotiation.cc:407] Connection header received
0924 07:17:47.608017 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:47.608018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:47.608029 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:47.608077 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:47.608983 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.609463 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.610354 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.610524 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.610565 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.612586 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:47.612588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:47.612588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:47.612595 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:47.613821 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:47.614172 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:47.614173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:47.614173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:47.614217 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:47.614398 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:47.614398 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:47.614399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:47.614486 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:47.614529 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:47.614738 (+ 209us) server_negotiation.cc:299] Negotiation successful
0924 07:17:47.614792 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260924 07:17:47.615036 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.607067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:43627 (local address 127.10.130.195:40405)
0924 07:17:47.607296 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:47.607341 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:47.607910 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:47.608138 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:47.608142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:47.608721 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:47.608904 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.608909 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.609522 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.609522 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.610294 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.610296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.610588 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.610589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.610682 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.611112 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:47.611120 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:47.612485 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:47.613892 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:47.613897 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:47.613898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:47.614108 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:47.614263 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:47.614264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:47.614264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:47.614353 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:47.614530 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:47.614532 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:47.614643 (+ 111us) client_negotiation.cc:769] Sending connection context
0924 07:17:47.614722 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 07:17:47.614833 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":88,"threads_started":1}
I20260924 07:17:47.615453 29249 heartbeater.cc:347] Connected to a master server at 127.10.130.254:43627
I20260924 07:17:47.615559 29249 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:47.615732 29249 heartbeater.cc:511] Master 127.10.130.254:43627 requested a full tablet report, sending...
I20260924 07:17:47.616104 28790 ts_manager.cc:194] Registered new tserver with Master: 021f3e552d3d4618ada6248999c50a6a (127.10.130.195:40599)
I20260924 07:17:47.616465 28790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:40405
I20260924 07:17:47.618462 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:17:47.625262 28848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.619608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32966 (local address 127.10.130.254:43627)
0924 07:17:47.619641 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:47.619643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:47.619811 (+ 168us) server_negotiation.cc:407] Connection header received
0924 07:17:47.619865 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:47.619866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:47.619876 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:47.619919 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:47.620452 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.620898 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.621604 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.621788 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.621838 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.622987 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:47.622989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:47.622989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:47.622996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:47.624211 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:47.624552 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:47.624553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:47.624553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:47.624599 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:47.624783 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:47.624784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:47.624784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:47.624903 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:47.624989 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:47.625123 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 07:17:47.625177 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:17:47.626693 28790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:17:47.628331 28790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:17:47.634690 29257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.630293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:38831 (local address 127.0.0.1:33890)
0924 07:17:47.630728 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:47.630737 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:47.630767 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:47.631028 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:47.631029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:47.631031 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:47.631083 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.631084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.632316 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.632317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.633769 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.633771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.634405 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.634407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.634481 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.634492 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 07:17:47.634558 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 07:17:47.634609 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":27,"spinlock_wait_cycles":384,"thread_start_us":82,"threads_started":1}
I20260924 07:17:47.634704 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.630386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33890 (local address 127.10.130.194:38831)
0924 07:17:47.630882 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:47.630885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:47.630895 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:17:47.630933 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:47.630935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:47.630951 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:47.631029 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:47.631129 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.632215 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.633837 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.634332 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.634396 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.634453 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:47.634585 (+ 132us) server_negotiation.cc:299] Negotiation successful
0924 07:17:47.634631 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":85,"threads_started":1}
I20260924 07:17:47.634838 29258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.630293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:35985 (local address 127.0.0.1:46358)
0924 07:17:47.630536 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:47.630571 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:47.630639 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:47.630867 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:47.630870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:47.630872 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:47.630936 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.630937 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.632316 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.632317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.633634 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.633637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.634594 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.634596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.634657 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.634662 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 07:17:47.634713 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 07:17:47.634753 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":52,"thread_start_us":52,"threads_started":1}
I20260924 07:17:47.635141 29262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.630530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46358 (local address 127.10.130.193:35985)
0924 07:17:47.630717 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:47.630721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:47.630729 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:17:47.630759 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:47.630761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:47.630773 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:47.630838 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:47.631059 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.632223 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.633719 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.634424 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.634500 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.634558 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:47.635004 (+ 446us) server_negotiation.cc:299] Negotiation successful
0924 07:17:47.635053 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":88,"threads_started":1}
I20260924 07:17:47.635435 29049 tablet_service.cc:1467] Processing CreateTablet for tablet e06a46d9a4be45108098546c00cadbc0 (DEFAULT_TABLE table=test-table [id=bfb5de992feb4c4ca92cb0bb94fe1798]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:47.635776 29049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06a46d9a4be45108098546c00cadbc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:47.635877 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.630300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:40599 (local address 127.0.0.1:38674)
0924 07:17:47.632371 (+ 2071us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:47.632381 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:47.632434 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:47.632599 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:47.632600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:47.632602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:47.632654 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.632656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.633604 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.633605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.634660 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.634662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.635641 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.635642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.635704 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.635709 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 07:17:47.635768 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 07:17:47.635805 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2053,"thread_start_us":48,"threads_started":1}
I20260924 07:17:47.635895 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.630499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38674 (local address 127.10.130.195:40599)
0924 07:17:47.630651 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:47.630656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:47.632448 (+ 1792us) server_negotiation.cc:407] Connection header received
0924 07:17:47.632499 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:47.632502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:47.632517 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:47.632593 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:47.632720 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.633511 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.634873 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.635555 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.635630 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.635679 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:47.635780 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 07:17:47.635817 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260924 07:17:47.636472 29183 tablet_service.cc:1467] Processing CreateTablet for tablet e06a46d9a4be45108098546c00cadbc0 (DEFAULT_TABLE table=test-table [id=bfb5de992feb4c4ca92cb0bb94fe1798]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:47.636708 29183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06a46d9a4be45108098546c00cadbc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:47.638393 29263 tablet_bootstrap.cc:492] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31: Bootstrap starting.
I20260924 07:17:47.638540 28915 tablet_service.cc:1467] Processing CreateTablet for tablet e06a46d9a4be45108098546c00cadbc0 (DEFAULT_TABLE table=test-table [id=bfb5de992feb4c4ca92cb0bb94fe1798]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:47.638767 28915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e06a46d9a4be45108098546c00cadbc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:47.639066 29263 tablet_bootstrap.cc:654] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:47.639395 29264 tablet_bootstrap.cc:492] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a: Bootstrap starting.
I20260924 07:17:47.639649 29263 log.cc:824] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:47.640021 29264 tablet_bootstrap.cc:654] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:47.640385 29263 tablet_bootstrap.cc:492] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31: No bootstrap required, opened a new log
I20260924 07:17:47.640435 29263 ts_tablet_manager.cc:1397] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:17:47.640527 29264 log.cc:824] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:47.641271 29264 tablet_bootstrap.cc:492] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a: No bootstrap required, opened a new log
I20260924 07:17:47.641342 29264 ts_tablet_manager.cc:1397] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:17:47.641657 29267 tablet_bootstrap.cc:492] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb: Bootstrap starting.
I20260924 07:17:47.642421 29267 tablet_bootstrap.cc:654] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:47.642381 29263 raft_consensus.cc:348] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e3045ed6e8041eaa357b35cc8006e31" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38831 } } peers { permanent_uuid: "021f3e552d3d4618ada6248999c50a6a" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40599 } } peers { permanent_uuid: "04108a981f994d909a017d8b146015eb" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 35985 } }
I20260924 07:17:47.642589 29263 raft_consensus.cc:374] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:47.642639 29263 raft_consensus.cc:729] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e3045ed6e8041eaa357b35cc8006e31, State: Initialized, Role: FOLLOWER
I20260924 07:17:47.642763 29263 consensus_queue.cc:260] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e3045ed6e8041eaa357b35cc8006e31" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38831 } } peers { permanent_uuid: "021f3e552d3d4618ada6248999c50a6a" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40599 } } peers { permanent_uuid: "04108a981f994d909a017d8b146015eb" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 35985 } }
I20260924 07:17:47.642978 29267 log.cc:824] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:47.643043 29263 ts_tablet_manager.cc:1428] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:17:47.643172 29115 heartbeater.cc:503] Master 127.10.130.254:43627 was elected leader, sending a full tablet report...
I20260924 07:17:47.643486 29264 raft_consensus.cc:348] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e3045ed6e8041eaa357b35cc8006e31" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38831 } } peers { permanent_uuid: "021f3e552d3d4618ada6248999c50a6a" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40599 } } peers { permanent_uuid: "04108a981f994d909a017d8b146015eb" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 35985 } }
I20260924 07:17:47.643639 29267 tablet_bootstrap.cc:492] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb: No bootstrap required, opened a new log
I20260924 07:17:47.643638 29264 raft_consensus.cc:374] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:47.643671 29264 raft_consensus.cc:729] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 021f3e552d3d4618ada6248999c50a6a, State: Initialized, Role: FOLLOWER
I20260924 07:17:47.643685 29267 ts_tablet_manager.cc:1397] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:17:47.643775 29264 consensus_queue.cc:260] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e3045ed6e8041eaa357b35cc8006e31" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38831 } } peers { permanent_uuid: "021f3e552d3d4618ada6248999c50a6a" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40599 } } peers { permanent_uuid: "04108a981f994d909a017d8b146015eb" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 35985 } }
I20260924 07:17:47.644069 29264 ts_tablet_manager.cc:1428] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 07:17:47.644086 29249 heartbeater.cc:503] Master 127.10.130.254:43627 was elected leader, sending a full tablet report...
I20260924 07:17:47.645349 29267 raft_consensus.cc:348] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e3045ed6e8041eaa357b35cc8006e31" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38831 } } peers { permanent_uuid: "021f3e552d3d4618ada6248999c50a6a" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40599 } } peers { permanent_uuid: "04108a981f994d909a017d8b146015eb" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 35985 } }
I20260924 07:17:47.645479 29267 raft_consensus.cc:374] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:47.645505 29267 raft_consensus.cc:729] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04108a981f994d909a017d8b146015eb, State: Initialized, Role: FOLLOWER
I20260924 07:17:47.645599 29267 consensus_queue.cc:260] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e3045ed6e8041eaa357b35cc8006e31" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38831 } } peers { permanent_uuid: "021f3e552d3d4618ada6248999c50a6a" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40599 } } peers { permanent_uuid: "04108a981f994d909a017d8b146015eb" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 35985 } }
I20260924 07:17:47.645859 29267 ts_tablet_manager.cc:1428] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:17:47.645937 28981 heartbeater.cc:503] Master 127.10.130.254:43627 was elected leader, sending a full tablet report...
W20260924 07:17:47.717358 29116 tablet.cc:2369] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:17:47.805272 29271 raft_consensus.cc:482] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:17:47.805399 29271 raft_consensus.cc:504] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e3045ed6e8041eaa357b35cc8006e31" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38831 } } peers { permanent_uuid: "021f3e552d3d4618ada6248999c50a6a" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40599 } } peers { permanent_uuid: "04108a981f994d909a017d8b146015eb" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 35985 } }
I20260924 07:17:47.805819 29271 leader_election.cc:288] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e3045ed6e8041eaa357b35cc8006e31 (127.10.130.194:38831), 021f3e552d3d4618ada6248999c50a6a (127.10.130.195:40599)
I20260924 07:17:47.808514 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.806079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:37767 (local address 127.10.130.195:40599)
0924 07:17:47.806121 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:47.806122 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:47.806132 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:17:47.806160 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:47.806160 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:47.806169 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:47.806216 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:47.806348 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.806815 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.807739 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.808208 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.808290 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.808301 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:47.808420 (+ 119us) server_negotiation.cc:299] Negotiation successful
0924 07:17:47.808449 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 07:17:47.808529 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.805997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:40599 (local address 127.10.130.193:37767)
0924 07:17:47.806049 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:47.806061 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:47.806100 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:47.806225 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:47.806226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:47.806227 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:47.806292 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.806294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.806882 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.806882 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.807660 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.807663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.808275 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.808276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.808356 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.808360 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:17:47.808415 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 07:17:47.808454 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 07:17:47.808758 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.806063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:36753 (local address 127.10.130.194:38831)
0924 07:17:47.806173 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:47.806175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:47.806187 (+ 12us) server_negotiation.cc:407] Connection header received
0924 07:17:47.806263 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:47.806264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:47.806272 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:47.806318 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:47.806450 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.806969 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.807878 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.808417 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.808472 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.808479 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:47.808651 (+ 172us) server_negotiation.cc:299] Negotiation successful
0924 07:17:47.808694 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260924 07:17:47.808861 29203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06a46d9a4be45108098546c00cadbc0" candidate_uuid: "04108a981f994d909a017d8b146015eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "021f3e552d3d4618ada6248999c50a6a" is_pre_election: true
I20260924 07:17:47.808934 29272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.806001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:38831 (local address 127.10.130.193:36753)
0924 07:17:47.806135 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:47.806144 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:47.806200 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:47.806328 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:47.806329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:47.806331 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:47.806397 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.806398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.807045 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.807046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.807795 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:47.807797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.808494 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.808495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:47.808564 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.808568 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:17:47.808827 (+ 259us) client_negotiation.cc:240] Negotiation successful
0924 07:17:47.808872 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":14,"thread_start_us":61,"threads_started":1}
I20260924 07:17:47.809026 29203 raft_consensus.cc:2454] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04108a981f994d909a017d8b146015eb in term 0.
I20260924 07:17:47.809238 28870 leader_election.cc:302] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 021f3e552d3d4618ada6248999c50a6a, 04108a981f994d909a017d8b146015eb; no voters:
I20260924 07:17:47.809270 29069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06a46d9a4be45108098546c00cadbc0" candidate_uuid: "04108a981f994d909a017d8b146015eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e3045ed6e8041eaa357b35cc8006e31" is_pre_election: true
I20260924 07:17:47.809360 29271 raft_consensus.cc:2781] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:17:47.809402 29069 raft_consensus.cc:2454] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04108a981f994d909a017d8b146015eb in term 0.
I20260924 07:17:47.809422 29271 raft_consensus.cc:482] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:17:47.809443 29271 raft_consensus.cc:3037] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:47.810264 29271 raft_consensus.cc:504] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e3045ed6e8041eaa357b35cc8006e31" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38831 } } peers { permanent_uuid: "021f3e552d3d4618ada6248999c50a6a" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40599 } } peers { permanent_uuid: "04108a981f994d909a017d8b146015eb" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 35985 } }
I20260924 07:17:47.810374 29271 leader_election.cc:288] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [CANDIDATE]: Term 1 election: Requested vote from peers 8e3045ed6e8041eaa357b35cc8006e31 (127.10.130.194:38831), 021f3e552d3d4618ada6248999c50a6a (127.10.130.195:40599)
I20260924 07:17:47.810501 29069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06a46d9a4be45108098546c00cadbc0" candidate_uuid: "04108a981f994d909a017d8b146015eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e3045ed6e8041eaa357b35cc8006e31"
I20260924 07:17:47.810568 29069 raft_consensus.cc:3037] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:47.810549 29203 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e06a46d9a4be45108098546c00cadbc0" candidate_uuid: "04108a981f994d909a017d8b146015eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "021f3e552d3d4618ada6248999c50a6a"
I20260924 07:17:47.810640 29203 raft_consensus.cc:3037] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:47.811214 29069 raft_consensus.cc:2454] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04108a981f994d909a017d8b146015eb in term 1.
I20260924 07:17:47.811389 29203 raft_consensus.cc:2454] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04108a981f994d909a017d8b146015eb in term 1.
I20260924 07:17:47.811378 28871 leader_election.cc:302] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04108a981f994d909a017d8b146015eb, 8e3045ed6e8041eaa357b35cc8006e31; no voters:
I20260924 07:17:47.811476 29271 raft_consensus.cc:2781] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:47.811618 29271 raft_consensus.cc:686] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [term 1 LEADER]: Becoming Leader. State: Replica: 04108a981f994d909a017d8b146015eb, State: Running, Role: LEADER
I20260924 07:17:47.811700 29271 consensus_queue.cc:237] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e3045ed6e8041eaa357b35cc8006e31" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38831 } } peers { permanent_uuid: "021f3e552d3d4618ada6248999c50a6a" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40599 } } peers { permanent_uuid: "04108a981f994d909a017d8b146015eb" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 35985 } }
I20260924 07:17:47.812475 28790 catalog_manager.cc:5495] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 04108a981f994d909a017d8b146015eb (127.10.130.193). New cstate: current_term: 1 leader_uuid: "04108a981f994d909a017d8b146015eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e3045ed6e8041eaa357b35cc8006e31" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "021f3e552d3d4618ada6248999c50a6a" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04108a981f994d909a017d8b146015eb" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 35985 } health_report { overall_health: HEALTHY } } }
W20260924 07:17:47.825842 28982 tablet.cc:2369] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 07:17:47.856904 29250 tablet.cc:2369] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:17:47.859230 29262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.857047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50886 (local address 127.10.130.193:35985)
0924 07:17:47.857088 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:47.857090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:47.857109 (+ 19us) server_negotiation.cc:407] Connection header received
0924 07:17:47.857173 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:47.857174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:47.857192 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:47.857242 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:17:47.857400 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.857871 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.858594 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:47.858787 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:47.858837 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:47.858941 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:17:47.859024 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:17:47.859087 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:47.859144 (+ 57us) server_negotiation.cc:299] Negotiation successful
0924 07:17:47.859154 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:17:47.860899 29203 raft_consensus.cc:1261] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a [term 1 FOLLOWER]: Refusing update from remote peer 04108a981f994d909a017d8b146015eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:47.860899 29069 raft_consensus.cc:1261] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31 [term 1 FOLLOWER]: Refusing update from remote peer 04108a981f994d909a017d8b146015eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:47.861208 29273 consensus_queue.cc:1035] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "021f3e552d3d4618ada6248999c50a6a" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 40599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:47.861353 29271 consensus_queue.cc:1035] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e3045ed6e8041eaa357b35cc8006e31" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 38831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:47.863862 29277 mvcc.cc:205] Tried to move back new op lower bound from 7332799561155862528 to 7332799560957358080. Current Snapshot: MvccSnapshot[applied={T|T < 7332799561155862528}]
I20260924 07:17:47.863940 29276 mvcc.cc:205] Tried to move back new op lower bound from 7332799561155862528 to 7332799560957358080. Current Snapshot: MvccSnapshot[applied={T|T < 7332799561155862528 or (T in {7332799561155862528})}]
I20260924 07:17:47.864094 29278 mvcc.cc:205] Tried to move back new op lower bound from 7332799561155862528 to 7332799560957358080. Current Snapshot: MvccSnapshot[applied={T|T < 7332799561155862528}]
I20260924 07:17:47.865435 28789 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:47.865516 28789 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:47.866211 28789 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70512f55626a4c158d2d7c74fac337b3: Sending DeleteTablet for 3 replicas of tablet e06a46d9a4be45108098546c00cadbc0
I20260924 07:17:47.866532 28915 tablet_service.cc:1514] Processing DeleteTablet for tablet e06a46d9a4be45108098546c00cadbc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:47 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:46358
I20260924 07:17:47.866549 29049 tablet_service.cc:1514] Processing DeleteTablet for tablet e06a46d9a4be45108098546c00cadbc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:47 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:33890
I20260924 07:17:47.866549 29183 tablet_service.cc:1514] Processing DeleteTablet for tablet e06a46d9a4be45108098546c00cadbc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:47 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:38674
I20260924 07:17:47.866757 29287 tablet_replica.cc:330] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb: stopping tablet replica
I20260924 07:17:47.866852 29287 raft_consensus.cc:2229] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:17:47.866937 29287 raft_consensus.cc:2258] T e06a46d9a4be45108098546c00cadbc0 P 04108a981f994d909a017d8b146015eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:47.867045 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 28852
I20260924 07:17:47.869912 29288 tablet_replica.cc:330] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a: stopping tablet replica
I20260924 07:17:47.870016 29288 raft_consensus.cc:2229] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:47.870118 29288 raft_consensus.cc:2258] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:47.870277 29286 tablet_replica.cc:330] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31: stopping tablet replica
I20260924 07:17:47.870370 29286 raft_consensus.cc:2229] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:47.870462 29286 raft_consensus.cc:2258] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:47.870568 29288 ts_tablet_manager.cc:1905] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:47.870801 29286 ts_tablet_manager.cc:1905] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:47.872404 29286 ts_tablet_manager.cc:1918] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:17:47.872426 29288 ts_tablet_manager.cc:1918] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:17:47.872466 29286 log.cc:1196] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/e06a46d9a4be45108098546c00cadbc0
I20260924 07:17:47.872483 29288 log.cc:1196] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/e06a46d9a4be45108098546c00cadbc0
I20260924 07:17:47.872716 29286 ts_tablet_manager.cc:1939] T e06a46d9a4be45108098546c00cadbc0 P 8e3045ed6e8041eaa357b35cc8006e31: Deleting consensus metadata
I20260924 07:17:47.872736 29288 ts_tablet_manager.cc:1939] T e06a46d9a4be45108098546c00cadbc0 P 021f3e552d3d4618ada6248999c50a6a: Deleting consensus metadata
I20260924 07:17:47.873077 28777 catalog_manager.cc:4841] TS 8e3045ed6e8041eaa357b35cc8006e31 (127.10.130.194:38831): tablet e06a46d9a4be45108098546c00cadbc0 (table test-table [id=bfb5de992feb4c4ca92cb0bb94fe1798]) successfully deleted
I20260924 07:17:47.873194 28776 catalog_manager.cc:4841] TS 021f3e552d3d4618ada6248999c50a6a (127.10.130.195:40599): tablet e06a46d9a4be45108098546c00cadbc0 (table test-table [id=bfb5de992feb4c4ca92cb0bb94fe1798]) successfully deleted
W20260924 07:17:47.873919 28774 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:35985 (error 108)
I20260924 07:17:47.874066 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 28986
I20260924 07:17:47.874639 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:47.874237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:35985 (local address 127.0.0.1:50892)
0924 07:17:47.874565 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:47.874606 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:35985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308}
W20260924 07:17:47.874758 28774 catalog_manager.cc:4584] TS 04108a981f994d909a017d8b146015eb (127.10.130.193:35985): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e06a46d9a4be45108098546c00cadbc0: Network error: Client connection negotiation failed: client connection to 127.10.130.193:35985: connect: Connection refused (error 111)
I20260924 07:17:47.878795 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 29120
I20260924 07:17:47.882540 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 28758
2026-09-24T07:17:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1925 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-24T07:17:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:17:47Z Disabled control of system clock
I20260924 07:17:47.907296 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:43329
--webserver_interface=127.10.130.254
--webserver_port=0
--builtin_ntp_servers=127.10.130.212:44477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:43329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 07:17:47.984567 29293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:47.984792 29293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:47.984812 29293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:47.985913 29293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:47.985949 29293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:47.985960 29293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:47.985970 29293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:47.985980 29293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:47.987248 29293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:43329
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:43329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:47.987440 29293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:47.987695 29293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:47.992136 29298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:47.992282 29293 server_base.cc:1034] running on GCE node
W20260924 07:17:47.992127 29301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:47.992110 29299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:47.992579 29293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:47.992866 29293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:47.994000 29293 hybrid_clock.cc:648] HybridClock initialized: now 1790234267993987 us; error 28 us; skew 500 ppm
I20260924 07:17:47.995155 29293 webserver.cc:460] Webserver started at http://127.10.130.254:37461/ using document root <none> and password file <none>
I20260924 07:17:47.995357 29293 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:47.995405 29293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:47.995530 29293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:47.996583 29293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "3d0ee3dc259f43a6b57fc41657671987"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "decd1c1e414a8bcb0ae08699277461a5"
server_key_iv: "c96e76fe4f908e8913433e27ecaddaa3"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.996904 29293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "3d0ee3dc259f43a6b57fc41657671987"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "decd1c1e414a8bcb0ae08699277461a5"
server_key_iv: "c96e76fe4f908e8913433e27ecaddaa3"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.998044 29293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:17:47.998646 29307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:47.998812 29293 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:47.998888 29293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "3d0ee3dc259f43a6b57fc41657671987"
format_stamp: "Formatted at 2026-09-24 07:17:47 on dist-test-slave-7q7m"
server_key: "decd1c1e414a8bcb0ae08699277461a5"
server_key_iv: "c96e76fe4f908e8913433e27ecaddaa3"
server_key_version: "encryptionkey@0"
I20260924 07:17:47.998944 29293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:48.013041 29293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:48.013348 29293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:48.013473 29293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:48.017156 29293 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:43329
I20260924 07:17:48.017194 29359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:43329 every 8 connection(s)
I20260924 07:17:48.017499 29293 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:48.018395 29360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:48.020668 29360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987: Bootstrap starting.
I20260924 07:17:48.021044 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 29293
I20260924 07:17:48.021118 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:17:48.021247 29360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:48.021334 10763 external_mini_cluster.cc:1428] Setting key f4e736346b60a1e120caacb30d5e4b8f
I20260924 07:17:48.021531 29360 log.cc:824] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:48.022207 29360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987: No bootstrap required, opened a new log
I20260924 07:17:48.024385 29360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0ee3dc259f43a6b57fc41657671987" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43329 } }
I20260924 07:17:48.024565 29360 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:48.024609 29360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0ee3dc259f43a6b57fc41657671987, State: Initialized, Role: FOLLOWER
I20260924 07:17:48.024740 29360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0ee3dc259f43a6b57fc41657671987" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43329 } }
I20260924 07:17:48.024823 29360 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:48.024865 29360 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:48.024926 29360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:48.025549 29360 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0ee3dc259f43a6b57fc41657671987" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43329 } }
I20260924 07:17:48.025672 29360 leader_election.cc:302] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d0ee3dc259f43a6b57fc41657671987; no voters:
I20260924 07:17:48.025844 29360 leader_election.cc:288] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:17:48.026022 29366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:48.026146 29366 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [term 1 LEADER]: Becoming Leader. State: Replica: 3d0ee3dc259f43a6b57fc41657671987, State: Running, Role: LEADER
I20260924 07:17:48.026157 29360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:48.026249 29366 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0ee3dc259f43a6b57fc41657671987" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43329 } }
I20260924 07:17:48.026757 29365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d0ee3dc259f43a6b57fc41657671987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0ee3dc259f43a6b57fc41657671987" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43329 } } }
I20260924 07:17:48.026872 29365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:48.026968 29364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.021783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42146 (local address 127.10.130.254:43329)
0924 07:17:48.021976 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:48.021983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:48.022008 (+ 25us) server_negotiation.cc:407] Connection header received
0924 07:17:48.022425 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:48.022431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:48.022489 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:48.022599 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:48.022822 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.023688 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.024845 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.025323 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.025409 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.026233 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:48.026238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:48.026238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:48.026416 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:48.026449 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:48.026460 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:48.026510 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:48.026612 (+ 102us) server_negotiation.cc:299] Negotiation successful
0924 07:17:48.026692 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
I20260924 07:17:48.027043 29367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d0ee3dc259f43a6b57fc41657671987. Latest consensus state: current_term: 1 leader_uuid: "3d0ee3dc259f43a6b57fc41657671987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d0ee3dc259f43a6b57fc41657671987" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 43329 } } }
I20260924 07:17:48.027108 29367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:48.027287 29374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:48.027717 29374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:48.029148 29374 catalog_manager.cc:1347] Generated new cluster ID: b3e6f7a1697747d2871c9e3ca90e6042
I20260924 07:17:48.029203 29374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:17:48.067366 29374 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:17:48.068130 29374 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:48.078538 29374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987: Generated new TSK 0
I20260924 07:17:48.078725 29374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:17:48.086577 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43329
--builtin_ntp_servers=127.10.130.212:44477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 07:17:48.188292 29384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:48.188489 29384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:48.188510 29384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:48.189596 29384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:48.189635 29384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:48.189661 29384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:17:48.190909 29384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:48.191107 29384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:48.191347 29384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:48.196039 29392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:48.196030 29389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:48.196013 29390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:48.196204 29384 server_base.cc:1034] running on GCE node
I20260924 07:17:48.196434 29384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:48.196666 29384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:48.197803 29384 hybrid_clock.cc:648] HybridClock initialized: now 1790234268197784 us; error 28 us; skew 500 ppm
I20260924 07:17:48.198980 29384 webserver.cc:460] Webserver started at http://127.10.130.193:33407/ using document root <none> and password file <none>
I20260924 07:17:48.199216 29384 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:48.199266 29384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:48.199391 29384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:48.200488 29384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "e6dd951a4bd44c53b4bbd219342912c8"
format_stamp: "Formatted at 2026-09-24 07:17:48 on dist-test-slave-7q7m"
server_key: "9cc76deca07c415aaaddea625856ee75"
server_key_iv: "61a72847d83890b060be54c98c884473"
server_key_version: "encryptionkey@0"
I20260924 07:17:48.200840 29384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "e6dd951a4bd44c53b4bbd219342912c8"
format_stamp: "Formatted at 2026-09-24 07:17:48 on dist-test-slave-7q7m"
server_key: "9cc76deca07c415aaaddea625856ee75"
server_key_iv: "61a72847d83890b060be54c98c884473"
server_key_version: "encryptionkey@0"
I20260924 07:17:48.202203 29384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:17:48.202904 29398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:48.203050 29384 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:48.203130 29384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "e6dd951a4bd44c53b4bbd219342912c8"
format_stamp: "Formatted at 2026-09-24 07:17:48 on dist-test-slave-7q7m"
server_key: "9cc76deca07c415aaaddea625856ee75"
server_key_iv: "61a72847d83890b060be54c98c884473"
server_key_version: "encryptionkey@0"
I20260924 07:17:48.203208 29384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:48.225109 29384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:48.225436 29384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:48.225574 29384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:48.225781 29384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:48.226090 29384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:48.226123 29384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:48.226198 29384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:48.226221 29384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:48.231679 29384 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:34711
I20260924 07:17:48.231750 29511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:34711 every 8 connection(s)
I20260924 07:17:48.232090 29384 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:17:48.237046 29364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.232814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:58381 (local address 127.10.130.254:43329)
0924 07:17:48.232871 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:48.232873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:48.233094 (+ 221us) server_negotiation.cc:407] Connection header received
0924 07:17:48.233579 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:48.233580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:48.233592 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:48.233648 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:48.234303 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.234856 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.235797 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.236011 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.236092 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.236819 (+ 727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:48.236821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:48.236821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:48.236826 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:48.236842 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:48.236846 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:48.236894 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:48.236954 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 07:17:48.236976 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 07:17:48.237219 29514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.232812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:43329 (local address 127.10.130.193:58381)
0924 07:17:48.232985 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:48.233019 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:48.233496 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:48.233704 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:48.233707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:48.233716 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:48.234199 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.234203 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.234930 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.234931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.235731 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.235733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.236094 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.236095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.236210 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.236694 (+ 484us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:17:48.236695 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:48.236722 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:17:48.236744 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:48.236896 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:48.236899 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:48.236901 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:17:48.236944 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 07:17:48.237020 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260924 07:17:48.237689 29512 heartbeater.cc:347] Connected to a master server at 127.10.130.254:43329
I20260924 07:17:48.237777 29512 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:48.237955 29512 heartbeater.cc:511] Master 127.10.130.254:43329 requested a full tablet report, sending...
I20260924 07:17:48.238411 29324 ts_manager.cc:194] Registered new tserver with Master: e6dd951a4bd44c53b4bbd219342912c8 (127.10.130.193:34711)
I20260924 07:17:48.238989 29324 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.130.193:58381
I20260924 07:17:48.242291 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 29384
I20260924 07:17:48.242388 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:17:48.242588 10763 external_mini_cluster.cc:1428] Setting key b6ed47c68a566b7080f7c048727cc45f
I20260924 07:17:48.243427 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43329
--builtin_ntp_servers=127.10.130.212:44477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 07:17:48.322109 29515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:48.322297 29515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:48.322317 29515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:48.323374 29515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:48.323407 29515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:48.323429 29515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:17:48.324721 29515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:48.324901 29515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:48.325139 29515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:48.329860 29521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:48.329854 29520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:48.329854 29523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:48.330054 29515 server_base.cc:1034] running on GCE node
I20260924 07:17:48.330286 29515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:48.330498 29515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:48.331627 29515 hybrid_clock.cc:648] HybridClock initialized: now 1790234268331616 us; error 33 us; skew 500 ppm
I20260924 07:17:48.332801 29515 webserver.cc:460] Webserver started at http://127.10.130.194:41825/ using document root <none> and password file <none>
I20260924 07:17:48.333011 29515 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:48.333058 29515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:48.333191 29515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:48.334172 29515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "704d1d62d47142ee85447935f43400d1"
format_stamp: "Formatted at 2026-09-24 07:17:48 on dist-test-slave-7q7m"
server_key: "a9918c3af18223be4f97f67c7754e3d1"
server_key_iv: "06396dd827e5a5032a5c230ce4585575"
server_key_version: "encryptionkey@0"
I20260924 07:17:48.334499 29515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "704d1d62d47142ee85447935f43400d1"
format_stamp: "Formatted at 2026-09-24 07:17:48 on dist-test-slave-7q7m"
server_key: "a9918c3af18223be4f97f67c7754e3d1"
server_key_iv: "06396dd827e5a5032a5c230ce4585575"
server_key_version: "encryptionkey@0"
I20260924 07:17:48.335618 29515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:17:48.336272 29529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:48.336413 29515 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:48.336493 29515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "704d1d62d47142ee85447935f43400d1"
format_stamp: "Formatted at 2026-09-24 07:17:48 on dist-test-slave-7q7m"
server_key: "a9918c3af18223be4f97f67c7754e3d1"
server_key_iv: "06396dd827e5a5032a5c230ce4585575"
server_key_version: "encryptionkey@0"
I20260924 07:17:48.336548 29515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:48.366382 29515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:48.366673 29515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:48.366796 29515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:48.366998 29515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:48.367311 29515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:48.367345 29515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:48.367373 29515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:48.367390 29515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:48.372591 29515 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:35381
I20260924 07:17:48.372635 29642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:35381 every 8 connection(s)
I20260924 07:17:48.372938 29515 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:17:48.377837 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 29515
I20260924 07:17:48.377883 29364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.373611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:46761 (local address 127.10.130.254:43329)
0924 07:17:48.373667 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:48.373669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:48.373953 (+ 284us) server_negotiation.cc:407] Connection header received
0924 07:17:48.374406 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:48.374407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:48.374419 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:48.374477 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:48.375122 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.375617 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.376553 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.376733 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.376778 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.377582 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:48.377584 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:48.377584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:48.377590 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:48.377611 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:48.377616 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:48.377686 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:48.377759 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 07:17:48.377791 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 07:17:48.377969 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:17:48.378016 29645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.373595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:43329 (local address 127.10.130.194:46761)
0924 07:17:48.373788 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:48.373830 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:48.374321 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:48.374541 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:48.374545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:48.374554 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:48.375040 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.375044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.375702 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.375703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.376497 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.376499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.376801 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.376802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.376915 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.377430 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:17:48.377431 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:48.377459 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:17:48.377483 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:48.377694 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:48.377697 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:48.377699 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:17:48.377744 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 07:17:48.377824 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260924 07:17:48.378183 10763 external_mini_cluster.cc:1428] Setting key 83bba610dba8099465bddc565d7ec9fb
I20260924 07:17:48.378572 29643 heartbeater.cc:347] Connected to a master server at 127.10.130.254:43329
I20260924 07:17:48.378675 29643 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:48.378872 29643 heartbeater.cc:511] Master 127.10.130.254:43329 requested a full tablet report, sending...
I20260924 07:17:48.379154 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43329
--builtin_ntp_servers=127.10.130.212:44477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 07:17:48.379245 29324 ts_manager.cc:194] Registered new tserver with Master: 704d1d62d47142ee85447935f43400d1 (127.10.130.194:35381)
I20260924 07:17:48.379599 29324 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.130.194:46761
W20260924 07:17:48.461136 29646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:48.461351 29646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:48.461375 29646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:48.462488 29646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:48.462538 29646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:48.462566 29646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:17:48.463919 29646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:43329
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:48.464175 29646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:48.464454 29646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:48.469051 29654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:48.469061 29652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:48.469152 29646 server_base.cc:1034] running on GCE node
W20260924 07:17:48.469059 29651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:48.469426 29646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:48.469642 29646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:48.470791 29646 hybrid_clock.cc:648] HybridClock initialized: now 1790234268470779 us; error 28 us; skew 500 ppm
I20260924 07:17:48.472013 29646 webserver.cc:460] Webserver started at http://127.10.130.195:46635/ using document root <none> and password file <none>
I20260924 07:17:48.472215 29646 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:48.472255 29646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:48.472344 29646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:48.473292 29646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "0ebf019d521b49dab58ec8c618161d26"
format_stamp: "Formatted at 2026-09-24 07:17:48 on dist-test-slave-7q7m"
server_key: "fe033e33bbf4df17c50af13e51fe15c2"
server_key_iv: "e3d285e5fd6edbe6edd15a01a27f2d19"
server_key_version: "encryptionkey@0"
I20260924 07:17:48.473567 29646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "0ebf019d521b49dab58ec8c618161d26"
format_stamp: "Formatted at 2026-09-24 07:17:48 on dist-test-slave-7q7m"
server_key: "fe033e33bbf4df17c50af13e51fe15c2"
server_key_iv: "e3d285e5fd6edbe6edd15a01a27f2d19"
server_key_version: "encryptionkey@0"
I20260924 07:17:48.474661 29646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:17:48.475234 29660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:48.475370 29646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:48.475437 29646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "0ebf019d521b49dab58ec8c618161d26"
format_stamp: "Formatted at 2026-09-24 07:17:48 on dist-test-slave-7q7m"
server_key: "fe033e33bbf4df17c50af13e51fe15c2"
server_key_iv: "e3d285e5fd6edbe6edd15a01a27f2d19"
server_key_version: "encryptionkey@0"
I20260924 07:17:48.475502 29646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:48.489324 29646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:48.489578 29646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:48.489696 29646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:48.489882 29646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:48.490168 29646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:48.490199 29646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:48.490219 29646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:48.490231 29646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:48.495281 29646 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:38211
I20260924 07:17:48.495347 29773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:38211 every 8 connection(s)
I20260924 07:17:48.495600 29646 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:17:48.500332 29364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.496252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:39235 (local address 127.10.130.254:43329)
0924 07:17:48.496304 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:48.496306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:48.496517 (+ 211us) server_negotiation.cc:407] Connection header received
0924 07:17:48.497005 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:48.497006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:48.497019 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:48.497069 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:48.497701 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.498182 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.499115 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.499305 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.499359 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.500069 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:48.500071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:48.500071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:48.500076 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:48.500092 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:48.500096 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:48.500136 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:48.500213 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 07:17:48.500256 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 07:17:48.500494 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.496241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:43329 (local address 127.10.130.195:39235)
0924 07:17:48.496410 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:48.496442 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:48.496911 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:48.497145 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:48.497149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:48.497159 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:48.497617 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.497622 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.498259 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.498260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.499037 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.499039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.499374 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.499376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.499494 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.499927 (+ 433us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:17:48.499927 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:48.499954 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:17:48.500001 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:48.500148 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:48.500153 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:48.500156 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:17:48.500212 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 07:17:48.500308 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260924 07:17:48.500902 29774 heartbeater.cc:347] Connected to a master server at 127.10.130.254:43329
I20260924 07:17:48.500981 29774 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:48.501168 29774 heartbeater.cc:511] Master 127.10.130.254:43329 requested a full tablet report, sending...
I20260924 07:17:48.501497 29324 ts_manager.cc:194] Registered new tserver with Master: 0ebf019d521b49dab58ec8c618161d26 (127.10.130.195:38211)
I20260924 07:17:48.501843 29324 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.130.195:39235
I20260924 07:17:48.503676 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 29646
I20260924 07:17:48.503772 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:17:48.503947 10763 external_mini_cluster.cc:1428] Setting key d429141991def53def20db147bd43fe8
I20260924 07:17:48.504662 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:17:48.508219 29364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.505800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42148 (local address 127.10.130.254:43329)
0924 07:17:48.505848 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:48.505849 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:48.505925 (+ 76us) server_negotiation.cc:407] Connection header received
0924 07:17:48.505980 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:48.505981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:48.505987 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:48.506035 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:48.506170 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.506599 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.507272 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.507435 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.507478 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.508050 (+ 572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:48.508051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:48.508051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:48.508055 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:48.508065 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:48.508068 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:48.508123 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:48.508163 (+ 40us) server_negotiation.cc:299] Negotiation successful
0924 07:17:48.508171 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 07:17:48.509394 29324 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:17:48.510856 29324 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:17:48.516603 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.513013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49218 (local address 127.10.130.195:38211)
0924 07:17:48.513156 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:48.513160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:48.513167 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:17:48.513194 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:48.513195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:48.513208 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:48.513284 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:48.513545 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.514298 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.515707 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.516232 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.516297 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.516375 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:48.516481 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 07:17:48.516515 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
I20260924 07:17:48.516623 29782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.512859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38211 (local address 127.0.0.1:49218)
0924 07:17:48.513018 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:48.513028 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:48.513072 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:48.513451 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:48.513452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:48.513453 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:48.513478 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.513480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.514382 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.514383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.515617 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.515620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.516311 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.516312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.516388 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.516401 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 07:17:48.516463 (+ 62us) client_negotiation.cc:240] Negotiation successful
0924 07:17:48.516515 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":24,"thread_start_us":57,"threads_started":1}
I20260924 07:17:48.516635 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.512670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:34711 (local address 127.0.0.1:60692)
0924 07:17:48.512809 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:48.512844 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:48.512904 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:48.513371 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:48.513372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:48.513373 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:48.513400 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.513401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.514650 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.514652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.515376 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.515378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.516381 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.516382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.516440 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.516444 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:17:48.516487 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 07:17:48.516528 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":2,"thread_start_us":70,"threads_started":1}
I20260924 07:17:48.516782 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.512707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:35381 (local address 127.0.0.1:35266)
0924 07:17:48.512968 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:48.512977 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:48.513023 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:48.513306 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:48.513309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:48.513311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:48.513375 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.513377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.514609 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.514610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.515678 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.515680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.516590 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.516591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.516645 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.516649 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:17:48.516694 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 07:17:48.516729 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":60,"threads_started":1}
I20260924 07:17:48.516894 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.512838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60692 (local address 127.10.130.193:34711)
0924 07:17:48.513186 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:48.513189 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:48.513197 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:17:48.513228 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:48.513229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:48.513241 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:48.513305 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:48.513472 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.514563 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.515465 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.516198 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.516264 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.516347 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:48.516784 (+ 437us) server_negotiation.cc:299] Negotiation successful
0924 07:17:48.516821 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":92,"threads_started":1}
I20260924 07:17:48.516980 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.512805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35266 (local address 127.10.130.194:35381)
0924 07:17:48.513135 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:48.513139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:48.513149 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:17:48.513190 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:48.513192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:48.513208 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:48.513284 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:48.513431 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.514530 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.515736 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.516428 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.516490 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.516559 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:48.516872 (+ 313us) server_negotiation.cc:299] Negotiation successful
0924 07:17:48.516909 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":77,"threads_started":1}
I20260924 07:17:48.517560 29577 tablet_service.cc:1467] Processing CreateTablet for tablet e1fb31f57eba4dc4a4a651a79d5694a0 (DEFAULT_TABLE table=test-table [id=fdbd9ffbff9148ff85b03a0fddff8b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:48.517560 29446 tablet_service.cc:1467] Processing CreateTablet for tablet e1fb31f57eba4dc4a4a651a79d5694a0 (DEFAULT_TABLE table=test-table [id=fdbd9ffbff9148ff85b03a0fddff8b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:48.517895 29446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1fb31f57eba4dc4a4a651a79d5694a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:48.517895 29577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1fb31f57eba4dc4a4a651a79d5694a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:48.520319 29708 tablet_service.cc:1467] Processing CreateTablet for tablet e1fb31f57eba4dc4a4a651a79d5694a0 (DEFAULT_TABLE table=test-table [id=fdbd9ffbff9148ff85b03a0fddff8b14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:48.520593 29708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1fb31f57eba4dc4a4a651a79d5694a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:48.521078 29788 tablet_bootstrap.cc:492] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8: Bootstrap starting.
I20260924 07:17:48.521276 29789 tablet_bootstrap.cc:492] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1: Bootstrap starting.
I20260924 07:17:48.521939 29789 tablet_bootstrap.cc:654] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:48.522262 29789 log.cc:824] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:48.522895 29789 tablet_bootstrap.cc:492] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1: No bootstrap required, opened a new log
I20260924 07:17:48.522958 29789 ts_tablet_manager.cc:1397] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:17:48.523075 29788 tablet_bootstrap.cc:654] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:48.523375 29788 log.cc:824] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:48.523659 29791 tablet_bootstrap.cc:492] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26: Bootstrap starting.
I20260924 07:17:48.524331 29791 tablet_bootstrap.cc:654] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:48.524452 29788 tablet_bootstrap.cc:492] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8: No bootstrap required, opened a new log
I20260924 07:17:48.524511 29788 ts_tablet_manager.cc:1397] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 07:17:48.524664 29791 log.cc:824] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:48.525354 29791 tablet_bootstrap.cc:492] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26: No bootstrap required, opened a new log
I20260924 07:17:48.525415 29791 ts_tablet_manager.cc:1397] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 07:17:48.525306 29789 raft_consensus.cc:348] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6dd951a4bd44c53b4bbd219342912c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 34711 } } peers { permanent_uuid: "704d1d62d47142ee85447935f43400d1" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 35381 } } peers { permanent_uuid: "0ebf019d521b49dab58ec8c618161d26" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38211 } }
I20260924 07:17:48.525453 29789 raft_consensus.cc:374] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:48.525492 29789 raft_consensus.cc:729] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 704d1d62d47142ee85447935f43400d1, State: Initialized, Role: FOLLOWER
I20260924 07:17:48.525615 29789 consensus_queue.cc:260] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6dd951a4bd44c53b4bbd219342912c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 34711 } } peers { permanent_uuid: "704d1d62d47142ee85447935f43400d1" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 35381 } } peers { permanent_uuid: "0ebf019d521b49dab58ec8c618161d26" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38211 } }
I20260924 07:17:48.525904 29789 ts_tablet_manager.cc:1428] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:17:48.525926 29643 heartbeater.cc:503] Master 127.10.130.254:43329 was elected leader, sending a full tablet report...
I20260924 07:17:48.526784 29788 raft_consensus.cc:348] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6dd951a4bd44c53b4bbd219342912c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 34711 } } peers { permanent_uuid: "704d1d62d47142ee85447935f43400d1" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 35381 } } peers { permanent_uuid: "0ebf019d521b49dab58ec8c618161d26" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38211 } }
I20260924 07:17:48.526975 29788 raft_consensus.cc:374] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:48.527069 29788 raft_consensus.cc:729] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6dd951a4bd44c53b4bbd219342912c8, State: Initialized, Role: FOLLOWER
I20260924 07:17:48.527184 29788 consensus_queue.cc:260] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6dd951a4bd44c53b4bbd219342912c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 34711 } } peers { permanent_uuid: "704d1d62d47142ee85447935f43400d1" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 35381 } } peers { permanent_uuid: "0ebf019d521b49dab58ec8c618161d26" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38211 } }
I20260924 07:17:48.527416 29788 ts_tablet_manager.cc:1428] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:17:48.527549 29512 heartbeater.cc:503] Master 127.10.130.254:43329 was elected leader, sending a full tablet report...
I20260924 07:17:48.527861 29791 raft_consensus.cc:348] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6dd951a4bd44c53b4bbd219342912c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 34711 } } peers { permanent_uuid: "704d1d62d47142ee85447935f43400d1" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 35381 } } peers { permanent_uuid: "0ebf019d521b49dab58ec8c618161d26" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38211 } }
I20260924 07:17:48.528038 29791 raft_consensus.cc:374] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:48.528095 29791 raft_consensus.cc:729] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ebf019d521b49dab58ec8c618161d26, State: Initialized, Role: FOLLOWER
I20260924 07:17:48.528203 29791 consensus_queue.cc:260] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6dd951a4bd44c53b4bbd219342912c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 34711 } } peers { permanent_uuid: "704d1d62d47142ee85447935f43400d1" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 35381 } } peers { permanent_uuid: "0ebf019d521b49dab58ec8c618161d26" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38211 } }
I20260924 07:17:48.528431 29791 ts_tablet_manager.cc:1428] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260924 07:17:48.528489 29774 heartbeater.cc:503] Master 127.10.130.254:43329 was elected leader, sending a full tablet report...
I20260924 07:17:48.541428 29795 raft_consensus.cc:482] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:17:48.541524 29795 raft_consensus.cc:504] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6dd951a4bd44c53b4bbd219342912c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 34711 } } peers { permanent_uuid: "704d1d62d47142ee85447935f43400d1" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 35381 } } peers { permanent_uuid: "0ebf019d521b49dab58ec8c618161d26" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38211 } }
I20260924 07:17:48.541818 29795 leader_election.cc:288] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 704d1d62d47142ee85447935f43400d1 (127.10.130.194:35381), 0ebf019d521b49dab58ec8c618161d26 (127.10.130.195:38211)
I20260924 07:17:48.544577 29514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.541967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:35381 (local address 127.10.130.193:51801)
0924 07:17:48.542009 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:48.542020 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:48.542058 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:48.542200 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:48.542201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:48.542203 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:48.542263 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.542265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.542923 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.542924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.543798 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.543801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.544366 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.544367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.544430 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.544435 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 07:17:48.544481 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 07:17:48.544523 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260924 07:17:48.544626 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.542020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:51801 (local address 127.10.130.194:35381)
0924 07:17:48.542072 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:48.542074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:48.542083 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:17:48.542117 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:48.542118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:48.542127 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:48.542190 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:48.542339 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.542855 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.543877 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.544299 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.544350 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.544356 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:48.544501 (+ 145us) server_negotiation.cc:299] Negotiation successful
0924 07:17:48.544541 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 07:17:48.544703 29797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.542069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38211 (local address 127.10.130.193:42241)
0924 07:17:48.542225 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:48.542232 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:48.542274 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:48.542378 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:48.542378 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:48.542380 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:48.542443 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.542444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.543033 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.543034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.543904 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:48.543906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.544460 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.544461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:48.544545 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.544549 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:17:48.544599 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 07:17:48.544635 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":9,"thread_start_us":96,"threads_started":1}
I20260924 07:17:48.544706 29787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.542122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:42241 (local address 127.10.130.195:38211)
0924 07:17:48.542166 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:48.542168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:48.542282 (+ 114us) server_negotiation.cc:407] Connection header received
0924 07:17:48.542307 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:48.542308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:48.542317 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:48.542359 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:48.542493 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.542964 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.544012 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.544386 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.544453 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.544460 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:48.544615 (+ 155us) server_negotiation.cc:299] Negotiation successful
0924 07:17:48.544651 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 07:17:48.544914 29728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1fb31f57eba4dc4a4a651a79d5694a0" candidate_uuid: "e6dd951a4bd44c53b4bbd219342912c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ebf019d521b49dab58ec8c618161d26" is_pre_election: true
I20260924 07:17:48.545001 29728 raft_consensus.cc:2454] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6dd951a4bd44c53b4bbd219342912c8 in term 0.
I20260924 07:17:48.545182 29401 leader_election.cc:302] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ebf019d521b49dab58ec8c618161d26, e6dd951a4bd44c53b4bbd219342912c8; no voters:
I20260924 07:17:48.545169 29597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1fb31f57eba4dc4a4a651a79d5694a0" candidate_uuid: "e6dd951a4bd44c53b4bbd219342912c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "704d1d62d47142ee85447935f43400d1" is_pre_election: true
I20260924 07:17:48.545290 29597 raft_consensus.cc:2454] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6dd951a4bd44c53b4bbd219342912c8 in term 0.
I20260924 07:17:48.545298 29795 raft_consensus.cc:2781] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:17:48.545367 29795 raft_consensus.cc:482] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:17:48.545389 29795 raft_consensus.cc:3037] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:48.546016 29795 raft_consensus.cc:504] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6dd951a4bd44c53b4bbd219342912c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 34711 } } peers { permanent_uuid: "704d1d62d47142ee85447935f43400d1" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 35381 } } peers { permanent_uuid: "0ebf019d521b49dab58ec8c618161d26" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38211 } }
I20260924 07:17:48.546106 29795 leader_election.cc:288] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [CANDIDATE]: Term 1 election: Requested vote from peers 704d1d62d47142ee85447935f43400d1 (127.10.130.194:35381), 0ebf019d521b49dab58ec8c618161d26 (127.10.130.195:38211)
I20260924 07:17:48.546260 29597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1fb31f57eba4dc4a4a651a79d5694a0" candidate_uuid: "e6dd951a4bd44c53b4bbd219342912c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "704d1d62d47142ee85447935f43400d1"
I20260924 07:17:48.546325 29597 raft_consensus.cc:3037] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:48.546972 29597 raft_consensus.cc:2454] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6dd951a4bd44c53b4bbd219342912c8 in term 1.
I20260924 07:17:48.547104 29401 leader_election.cc:302] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 704d1d62d47142ee85447935f43400d1, e6dd951a4bd44c53b4bbd219342912c8; no voters:
I20260924 07:17:48.547097 29728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1fb31f57eba4dc4a4a651a79d5694a0" candidate_uuid: "e6dd951a4bd44c53b4bbd219342912c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ebf019d521b49dab58ec8c618161d26"
I20260924 07:17:48.547159 29728 raft_consensus.cc:3037] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:48.547219 29795 raft_consensus.cc:2781] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:48.547355 29795 raft_consensus.cc:686] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [term 1 LEADER]: Becoming Leader. State: Replica: e6dd951a4bd44c53b4bbd219342912c8, State: Running, Role: LEADER
I20260924 07:17:48.547444 29795 consensus_queue.cc:237] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6dd951a4bd44c53b4bbd219342912c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 34711 } } peers { permanent_uuid: "704d1d62d47142ee85447935f43400d1" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 35381 } } peers { permanent_uuid: "0ebf019d521b49dab58ec8c618161d26" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38211 } }
I20260924 07:17:48.547904 29728 raft_consensus.cc:2454] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6dd951a4bd44c53b4bbd219342912c8 in term 1.
I20260924 07:17:48.548166 29324 catalog_manager.cc:5495] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6dd951a4bd44c53b4bbd219342912c8 (127.10.130.193). New cstate: current_term: 1 leader_uuid: "e6dd951a4bd44c53b4bbd219342912c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6dd951a4bd44c53b4bbd219342912c8" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 34711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "704d1d62d47142ee85447935f43400d1" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 35381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ebf019d521b49dab58ec8c618161d26" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38211 } health_report { overall_health: UNKNOWN } } }
I20260924 07:17:48.554406 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.552427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60706 (local address 127.10.130.193:34711)
0924 07:17:48.552480 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:48.552482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:48.552488 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:17:48.552512 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:48.552512 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:48.552527 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:48.552580 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:17:48.552674 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.553087 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.553819 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:48.553997 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:48.554053 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:48.554162 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:17:48.554240 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:17:48.554303 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:48.554339 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 07:17:48.554346 (+ 7us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 07:17:48.555912 29728 raft_consensus.cc:1261] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26 [term 1 FOLLOWER]: Refusing update from remote peer e6dd951a4bd44c53b4bbd219342912c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:48.555936 29597 raft_consensus.cc:1261] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1 [term 1 FOLLOWER]: Refusing update from remote peer e6dd951a4bd44c53b4bbd219342912c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:48.556244 29795 consensus_queue.cc:1035] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "704d1d62d47142ee85447935f43400d1" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 35381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:48.556382 29798 consensus_queue.cc:1035] T e1fb31f57eba4dc4a4a651a79d5694a0 P e6dd951a4bd44c53b4bbd219342912c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ebf019d521b49dab58ec8c618161d26" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:48.559191 29803 mvcc.cc:205] Tried to move back new op lower bound from 7332799564002852864 to 7332799563970834432. Current Snapshot: MvccSnapshot[applied={T|T < 7332799564002852864}]
I20260924 07:17:48.561019 29324 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:48.561102 29324 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:48.561774 29324 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d0ee3dc259f43a6b57fc41657671987: Sending DeleteTablet for 3 replicas of tablet e1fb31f57eba4dc4a4a651a79d5694a0
I20260924 07:17:48.562073 29446 tablet_service.cc:1514] Processing DeleteTablet for tablet e1fb31f57eba4dc4a4a651a79d5694a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:48 UTC) from {username='slave'} at 127.0.0.1:60692
I20260924 07:17:48.562112 29577 tablet_service.cc:1514] Processing DeleteTablet for tablet e1fb31f57eba4dc4a4a651a79d5694a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:48 UTC) from {username='slave'} at 127.0.0.1:35266
I20260924 07:17:48.562264 29708 tablet_service.cc:1514] Processing DeleteTablet for tablet e1fb31f57eba4dc4a4a651a79d5694a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:48 UTC) from {username='slave'} at 127.0.0.1:49218
I20260924 07:17:48.562317 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 29384
I20260924 07:17:48.564306 29813 tablet_replica.cc:330] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26: stopping tablet replica
I20260924 07:17:48.564393 29813 raft_consensus.cc:2229] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:48.564491 29813 raft_consensus.cc:2258] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:48.564858 29813 ts_tablet_manager.cc:1905] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:48.566380 29813 ts_tablet_manager.cc:1918] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:17:48.566431 29813 log.cc:1196] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790234044967003-10763-0/minicluster-data/ts-2/wal/wals/e1fb31f57eba4dc4a4a651a79d5694a0
I20260924 07:17:48.566609 29813 ts_tablet_manager.cc:1939] T e1fb31f57eba4dc4a4a651a79d5694a0 P 0ebf019d521b49dab58ec8c618161d26: Deleting consensus metadata
I20260924 07:17:48.566952 29310 catalog_manager.cc:4841] TS 0ebf019d521b49dab58ec8c618161d26 (127.10.130.195:38211): tablet e1fb31f57eba4dc4a4a651a79d5694a0 (table test-table [id=fdbd9ffbff9148ff85b03a0fddff8b14]) successfully deleted
W20260924 07:17:48.567425 29309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:34711 (error 108)
I20260924 07:17:48.567556 29812 tablet_replica.cc:330] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1: stopping tablet replica
I20260924 07:17:48.567654 29812 raft_consensus.cc:2229] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:48.567751 29812 raft_consensus.cc:2258] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:48.567847 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.567744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:34711 (local address 127.0.0.1:60708)
0924 07:17:48.567780 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:48.567816 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:34711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 07:17:48.567912 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 29515
I20260924 07:17:48.568152 29812 ts_tablet_manager.cc:1905] T e1fb31f57eba4dc4a4a651a79d5694a0 P 704d1d62d47142ee85447935f43400d1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 07:17:48.569038 29309 catalog_manager.cc:4584] TS e6dd951a4bd44c53b4bbd219342912c8 (127.10.130.193:34711): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1fb31f57eba4dc4a4a651a79d5694a0: Network error: Client connection negotiation failed: client connection to 127.10.130.193:34711: connect: Connection refused (error 111)
I20260924 07:17:48.571463 29783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:48.571357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:35381 (local address 127.0.0.1:35272)
0924 07:17:48.571392 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:48.571436 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.194:35381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260924 07:17:48.572220 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 29646
I20260924 07:17:48.576340 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 29293
2026-09-24T07:17:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (693 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:17:48 dist-test-slave-7q7m krb5kdc[29817](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:17:48 dist-test-slave-7q7m krb5kdc[29817](info): set up 2 sockets
Sep 24 07:17:48 dist-test-slave-7q7m krb5kdc[29817](info): commencing operation
krb5kdc: starting...
W20260924 07:17:49.512288 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.918s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:17:49 dist-test-slave-7q7m krb5kdc[29817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:17:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:17:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:17:49.626055 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39093
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:38369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:39093
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:49.704246 29830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:49.704447 29830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:49.704476 29830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:49.705605 29830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:49.705646 29830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:49.705663 29830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:49.705684 29830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:49.705700 29830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:49.707100 29830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:39093
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:49.707329 29830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:49.707525 29830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:49.711830 29838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:49.711838 29836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:49.712110 29830 server_base.cc:1034] running on GCE node
W20260924 07:17:49.711841 29835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:49.712368 29830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:49.712644 29830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:49.713774 29830 hybrid_clock.cc:648] HybridClock initialized: now 1790234269713760 us; error 28 us; skew 500 ppm
Sep 24 07:17:49 dist-test-slave-7q7m krb5kdc[29817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234269, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:49.721750 29830 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:17:49.722131 29830 webserver.cc:460] Webserver started at http://127.10.130.254:45023/ using document root <none> and password file <none>
I20260924 07:17:49.722291 29830 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:49.722328 29830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:49.722411 29830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:49.723347 29830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "27bf8862c1404961a93fcc680eba948e"
format_stamp: "Formatted at 2026-09-24 07:17:49 on dist-test-slave-7q7m"
server_key: "1dc7a64be65e460a0e5cc6cea5e3d3de"
server_key_iv: "0331aef803446c6dddf64e35db708198"
server_key_version: "encryptionkey@0"
I20260924 07:17:49.723626 29830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "27bf8862c1404961a93fcc680eba948e"
format_stamp: "Formatted at 2026-09-24 07:17:49 on dist-test-slave-7q7m"
server_key: "1dc7a64be65e460a0e5cc6cea5e3d3de"
server_key_iv: "0331aef803446c6dddf64e35db708198"
server_key_version: "encryptionkey@0"
I20260924 07:17:49.724797 29830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:17:49.725409 29845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:49.725561 29830 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:17:49.725633 29830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "27bf8862c1404961a93fcc680eba948e"
format_stamp: "Formatted at 2026-09-24 07:17:49 on dist-test-slave-7q7m"
server_key: "1dc7a64be65e460a0e5cc6cea5e3d3de"
server_key_iv: "0331aef803446c6dddf64e35db708198"
server_key_version: "encryptionkey@0"
I20260924 07:17:49.725708 29830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:49.740193 29830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:49.742056 29830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:49.742174 29830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:49.745586 29830 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:39093
I20260924 07:17:49.745628 29897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:39093 every 8 connection(s)
I20260924 07:17:49.745916 29830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:49.746632 29898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:49.748790 29898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e: Bootstrap starting.
I20260924 07:17:49.749326 29898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:49.749578 29898 log.cc:824] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:49.750005 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 29830
I20260924 07:17:49.750128 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:17:49.750319 10763 external_mini_cluster.cc:1428] Setting key 37ed8c61cc746c202476ece48fc9f9f4
I20260924 07:17:49.750319 29898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e: No bootstrap required, opened a new log
I20260924 07:17:49.752425 29898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27bf8862c1404961a93fcc680eba948e" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39093 } }
I20260924 07:17:49.752555 29898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:49.752579 29898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27bf8862c1404961a93fcc680eba948e, State: Initialized, Role: FOLLOWER
I20260924 07:17:49.752684 29898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27bf8862c1404961a93fcc680eba948e" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39093 } }
I20260924 07:17:49.752748 29898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:49.752768 29898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:49.752812 29898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:49.753334 29898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27bf8862c1404961a93fcc680eba948e" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39093 } }
I20260924 07:17:49.753419 29898 leader_election.cc:302] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27bf8862c1404961a93fcc680eba948e; no voters:
I20260924 07:17:49.753561 29898 leader_election.cc:288] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:17:49.753599 29903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:49.753706 29903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [term 1 LEADER]: Becoming Leader. State: Replica: 27bf8862c1404961a93fcc680eba948e, State: Running, Role: LEADER
I20260924 07:17:49.753793 29898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:49.753782 29903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27bf8862c1404961a93fcc680eba948e" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39093 } }
I20260924 07:17:49.754086 29903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27bf8862c1404961a93fcc680eba948e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27bf8862c1404961a93fcc680eba948e" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39093 } } }
I20260924 07:17:49.754161 29903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:49.754272 29903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27bf8862c1404961a93fcc680eba948e. Latest consensus state: current_term: 1 leader_uuid: "27bf8862c1404961a93fcc680eba948e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27bf8862c1404961a93fcc680eba948e" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39093 } } }
I20260924 07:17:49.754335 29903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:49.754462 29908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:49.754971 29908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:49.757181 29908 catalog_manager.cc:1347] Generated new cluster ID: 9304445b65524644b3da40711a613e28
I20260924 07:17:49.757232 29908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:17:49 dist-test-slave-7q7m krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:49.761678 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:49.750782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52846 (local address 127.10.130.254:39093)
0924 07:17:49.750950 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:49.750958 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:49.750981 (+ 23us) server_negotiation.cc:407] Connection header received
0924 07:17:49.751500 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:49.751507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:49.751550 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:49.751672 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:49.752398 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:49.753636 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:49.754547 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:49.755043 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:49.755149 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:49.758179 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:49.758184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:49.758185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:49.758196 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:49.760124 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:49.760470 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:49.760473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:49.760473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:49.760581 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:49.760747 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:49.760748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:49.760749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:49.761050 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:49.761232 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:49.761330 (+ 98us) server_negotiation.cc:299] Negotiation successful
0924 07:17:49.761477 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260924 07:17:49.781108 29908 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:17:49.781843 29908 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:49.785635 29908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e: Generated new TSK 0
I20260924 07:17:49.785801 29908 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.130.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
Entry for principal kudu/127.10.130.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab.
I20260924 07:17:49.811440 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39093
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:38369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:49.889158 29924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:49.889348 29924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:49.889369 29924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:49.890470 29924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:49.890508 29924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:49.890534 29924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:17:49.891916 29924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.193.keytab
--principal=kudu/127.10.130.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:49.892153 29924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:49.892351 29924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:49.896910 29929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:49.896921 29932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:49.897082 29924 server_base.cc:1034] running on GCE node
W20260924 07:17:49.897151 29930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:49.897338 29924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:49.897529 29924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:49.898660 29924 hybrid_clock.cc:648] HybridClock initialized: now 1790234269898647 us; error 30 us; skew 500 ppm
Sep 24 07:17:49 dist-test-slave-7q7m krb5kdc[29817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234269, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:49.900888 29924 init.cc:377] Logged in from keytab as kudu/127.10.130.193@KRBTEST.COM (short username kudu)
I20260924 07:17:49.901257 29924 webserver.cc:460] Webserver started at http://127.10.130.193:39291/ using document root <none> and password file <none>
I20260924 07:17:49.901450 29924 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:49.901491 29924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:49.901609 29924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:49.902529 29924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "842a057bf30b42108da045df4d37f344"
format_stamp: "Formatted at 2026-09-24 07:17:49 on dist-test-slave-7q7m"
server_key: "7709825c4ef1cdb1c041075597188fb3"
server_key_iv: "3a8c8066c87e79f56dbea803a56e97fe"
server_key_version: "encryptionkey@0"
I20260924 07:17:49.902848 29924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "842a057bf30b42108da045df4d37f344"
format_stamp: "Formatted at 2026-09-24 07:17:49 on dist-test-slave-7q7m"
server_key: "7709825c4ef1cdb1c041075597188fb3"
server_key_iv: "3a8c8066c87e79f56dbea803a56e97fe"
server_key_version: "encryptionkey@0"
I20260924 07:17:49.903970 29924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:17:49.904611 29939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:49.904753 29924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:49.904826 29924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "842a057bf30b42108da045df4d37f344"
format_stamp: "Formatted at 2026-09-24 07:17:49 on dist-test-slave-7q7m"
server_key: "7709825c4ef1cdb1c041075597188fb3"
server_key_iv: "3a8c8066c87e79f56dbea803a56e97fe"
server_key_version: "encryptionkey@0"
I20260924 07:17:49.904892 29924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:49.912636 29924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:49.914124 29924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:49.914233 29924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:49.914407 29924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:49.914664 29924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:49.914698 29924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:49.914717 29924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:49.914739 29924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:49.919898 29924 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:38967
I20260924 07:17:49.919957 30052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:38967 every 8 connection(s)
I20260924 07:17:49.920218 29924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:17:49.925200 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 29924
I20260924 07:17:49.925314 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:17:49.925505 10763 external_mini_cluster.cc:1428] Setting key 5d23a87664dbe79bea6b2d7fbd32a599
Sep 24 07:17:49 dist-test-slave-7q7m krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234269, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.193@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:49.928642 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:49.920845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:41609 (local address 127.10.130.254:39093)
0924 07:17:49.920902 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:49.920904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:49.921065 (+ 161us) server_negotiation.cc:407] Connection header received
0924 07:17:49.921530 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:49.921531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:49.921539 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:49.921575 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:49.922403 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:49.922889 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:49.923772 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:49.923957 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:49.924045 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:49.926072 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:49.926073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:49.926074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:49.926082 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:49.927287 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:49.927682 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:49.927683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:49.927683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:49.927742 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:49.927993 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:49.927994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:49.927995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:49.928117 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:49.928181 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:49.928451 (+ 270us) server_negotiation.cc:299] Negotiation successful
0924 07:17:49.928545 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 07:17:49.928960 30055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:49.920821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39093 (local address 127.10.130.193:41609)
0924 07:17:49.920956 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:49.920993 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:49.921441 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:49.921639 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:49.921642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:49.922157 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:49.922334 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:49.922339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:49.922945 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:49.922946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:49.923716 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:49.923718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:49.924061 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:49.924061 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:49.924158 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:49.924520 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:49.924527 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:49.925959 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:49.927362 (+ 1403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:49.927369 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:49.927371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:49.927609 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:49.927801 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:49.927802 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:49.927802 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:49.927910 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:49.928180 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:49.928187 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:49.928331 (+ 144us) client_negotiation.cc:769] Sending connection context
0924 07:17:49.928424 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 07:17:49.928573 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":71,"threads_started":1}
I20260924 07:17:49.929419 30053 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39093
I20260924 07:17:49.929507 30053 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:49.929672 30053 heartbeater.cc:511] Master 127.10.130.254:39093 requested a full tablet report, sending...
I20260924 07:17:49.930210 29862 ts_manager.cc:194] Registered new tserver with Master: 842a057bf30b42108da045df4d37f344 (127.10.130.193:38967)
I20260924 07:17:49.931044 29862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.193@KRBTEST.COM'} at 127.10.130.193:41609
WARNING: no policy specified for kudu/127.10.130.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
Entry for principal kudu/127.10.130.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab.
I20260924 07:17:49.946779 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39093
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:38369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:50.023388 30058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:50.023589 30058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:50.023612 30058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:50.024703 30058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:50.024734 30058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:50.024758 30058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:17:50.026607 30058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.194.keytab
--principal=kudu/127.10.130.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:50.026804 30058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:50.026978 30058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:50.031626 30066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:50.031654 30064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:50.031601 30063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:50.031751 30058 server_base.cc:1034] running on GCE node
I20260924 07:17:50.031919 30058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:50.032156 30058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:50.033277 30058 hybrid_clock.cc:648] HybridClock initialized: now 1790234270033269 us; error 25 us; skew 500 ppm
Sep 24 07:17:50 dist-test-slave-7q7m krb5kdc[29817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234270, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:50.035567 30058 init.cc:377] Logged in from keytab as kudu/127.10.130.194@KRBTEST.COM (short username kudu)
I20260924 07:17:50.035939 30058 webserver.cc:460] Webserver started at http://127.10.130.194:43399/ using document root <none> and password file <none>
I20260924 07:17:50.036146 30058 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:50.036195 30058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:50.036314 30058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:50.037268 30058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "9648a4784a5e4f8797ca367139731feb"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "fa055eecd3a4366768a74f5252259b62"
server_key_iv: "a1e3311520031039f2e781f68a483d49"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.037572 30058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "9648a4784a5e4f8797ca367139731feb"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "fa055eecd3a4366768a74f5252259b62"
server_key_iv: "a1e3311520031039f2e781f68a483d49"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.038733 30058 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:17:50.039321 30073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.039459 30058 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:50.039537 30058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "9648a4784a5e4f8797ca367139731feb"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "fa055eecd3a4366768a74f5252259b62"
server_key_iv: "a1e3311520031039f2e781f68a483d49"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.039590 30058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:50.054299 30058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:50.056818 30058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:50.056965 30058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:50.057185 30058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:50.057507 30058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:50.057540 30058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.057574 30058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:50.057590 30058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.062796 30058 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:34027
I20260924 07:17:50.062839 30186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:34027 every 8 connection(s)
I20260924 07:17:50.063117 30058 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
Sep 24 07:17:50 dist-test-slave-7q7m krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234270, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.194@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:50.070755 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 30058
I20260924 07:17:50.070863 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:17:50.071045 10763 external_mini_cluster.cc:1428] Setting key d02f74c6f98e1c4d428d6578780fb148
I20260924 07:17:50.071208 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.063743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:34863 (local address 127.10.130.254:39093)
0924 07:17:50.063794 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.063796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.063966 (+ 170us) server_negotiation.cc:407] Connection header received
0924 07:17:50.064475 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.064477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.064489 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.064545 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:50.065410 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.065910 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.066847 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.067048 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.067103 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.068993 (+ 1890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:50.068994 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:50.068994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:50.069001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:50.070058 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:50.070378 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:50.070379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:50.070379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:50.070422 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:50.070597 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:50.070598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:50.070598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:50.070690 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:50.070733 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.071045 (+ 312us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.071113 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 07:17:50.071384 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.063728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39093 (local address 127.10.130.194:34863)
0924 07:17:50.063861 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.063894 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:50.064384 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:50.064617 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:50.064620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:50.065141 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:50.065321 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.065326 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.065987 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.065988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.066768 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.066770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.067117 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.067118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.067228 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.067652 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:50.067660 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:50.068908 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:50.070122 (+ 1214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:50.070127 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:50.070129 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:50.070324 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:50.070456 (+ 132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:50.070457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:50.070457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:50.070535 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:50.070742 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:50.070748 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:50.070905 (+ 157us) client_negotiation.cc:769] Sending connection context
0924 07:17:50.071017 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 07:17:50.071179 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
I20260924 07:17:50.071802 30187 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39093
I20260924 07:17:50.071894 30187 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:50.072084 30187 heartbeater.cc:511] Master 127.10.130.254:39093 requested a full tablet report, sending...
I20260924 07:17:50.072438 29861 ts_manager.cc:194] Registered new tserver with Master: 9648a4784a5e4f8797ca367139731feb (127.10.130.194:34027)
I20260924 07:17:50.072790 29861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.194@KRBTEST.COM'} at 127.10.130.194:34863
WARNING: no policy specified for kudu/127.10.130.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
Entry for principal kudu/127.10.130.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab.
I20260924 07:17:50.092374 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39093
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:38369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:50.169229 30192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:50.169425 30192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:50.169447 30192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:50.170538 30192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:50.170572 30192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:50.170598 30192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:17:50.172168 30192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.195.keytab
--principal=kudu/127.10.130.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:50.172394 30192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:50.172632 30192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:50.177174 30197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:50.177191 30198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:50.177191 30200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:50.177433 30192 server_base.cc:1034] running on GCE node
I20260924 07:17:50.177624 30192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:50.177842 30192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:50.178974 30192 hybrid_clock.cc:648] HybridClock initialized: now 1790234270178966 us; error 29 us; skew 500 ppm
Sep 24 07:17:50 dist-test-slave-7q7m krb5kdc[29817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234270, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:50.181318 30192 init.cc:377] Logged in from keytab as kudu/127.10.130.195@KRBTEST.COM (short username kudu)
I20260924 07:17:50.181725 30192 webserver.cc:460] Webserver started at http://127.10.130.195:37003/ using document root <none> and password file <none>
I20260924 07:17:50.181908 30192 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:50.181953 30192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:50.182071 30192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:50.183027 30192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "f44e83bf4757469680045bb28f835be8"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "c0d598a06ac589fbb430063a9e76fae8"
server_key_iv: "a4f0613ce3d380b72b6b87f40cf9b7aa"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.183350 30192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "f44e83bf4757469680045bb28f835be8"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "c0d598a06ac589fbb430063a9e76fae8"
server_key_iv: "a4f0613ce3d380b72b6b87f40cf9b7aa"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.184563 30192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:17:50.185166 30207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.185289 30192 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:50.185365 30192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "f44e83bf4757469680045bb28f835be8"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "c0d598a06ac589fbb430063a9e76fae8"
server_key_iv: "a4f0613ce3d380b72b6b87f40cf9b7aa"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.185422 30192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:50.210489 30192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:50.212685 30192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:50.212822 30192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:50.213035 30192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:50.213338 30192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:50.213370 30192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.213399 30192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:50.213420 30192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.218611 30192 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:37473
I20260924 07:17:50.218668 30320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:37473 every 8 connection(s)
I20260924 07:17:50.218947 30192 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
Sep 24 07:17:50 dist-test-slave-7q7m krb5kdc[29817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234270, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.195@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:17:50.226508 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 30192
I20260924 07:17:50.226621 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:17:50.226787 10763 external_mini_cluster.cc:1428] Setting key eaffb28a40efa3d19e1a2c10b45cd0c2
I20260924 07:17:50.227135 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.219575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:44439 (local address 127.10.130.254:39093)
0924 07:17:50.219637 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.219639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.219840 (+ 201us) server_negotiation.cc:407] Connection header received
0924 07:17:50.220325 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.220326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.220335 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.220387 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:50.221293 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.221797 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.222707 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.222881 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.222922 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.224814 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:50.224816 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:50.224816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:50.224822 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:50.225912 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:50.226247 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:50.226248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:50.226248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:50.226292 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:50.226532 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:50.226533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:50.226533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:50.226642 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:50.226701 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.226958 (+ 257us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.227034 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 07:17:50.227290 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.219538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39093 (local address 127.10.130.195:44439)
0924 07:17:50.219720 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.219761 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:50.220248 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:50.220449 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:50.220452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:50.220994 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:50.221197 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.221202 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.221862 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.221862 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.222647 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.222649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.222932 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.222933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.223023 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.223427 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:17:50.223434 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:50.224736 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:50.225975 (+ 1239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:50.225981 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:50.225982 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:50.226194 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:50.226353 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:17:50.226353 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:17:50.226354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:17:50.226477 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:17:50.226698 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:50.226705 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:50.226833 (+ 128us) client_negotiation.cc:769] Sending connection context
0924 07:17:50.226937 (+ 104us) client_negotiation.cc:240] Negotiation successful
0924 07:17:50.227084 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260924 07:17:50.227703 30321 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39093
I20260924 07:17:50.227795 30321 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:50.228005 30321 heartbeater.cc:511] Master 127.10.130.254:39093 requested a full tablet report, sending...
I20260924 07:17:50.228341 29861 ts_manager.cc:194] Registered new tserver with Master: f44e83bf4757469680045bb28f835be8 (127.10.130.195:37473)
I20260924 07:17:50.228688 29861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.130.195@KRBTEST.COM'} at 127.10.130.195:44439
I20260924 07:17:50.238462 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:17:50.244897 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.239630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52848 (local address 127.10.130.254:39093)
0924 07:17:50.239663 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.239665 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.239783 (+ 118us) server_negotiation.cc:407] Connection header received
0924 07:17:50.239835 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.239836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.239845 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.239885 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:50.240396 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.240867 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.241543 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.241721 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.241761 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.242784 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:50.242785 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:50.242785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:50.242791 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:50.244052 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:50.244349 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:50.244349 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:50.244350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:50.244396 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:50.244506 (+ 110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:50.244507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:50.244507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:50.244641 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:50.244707 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.244782 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.244836 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:17:50.246250 29861 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52848:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:17:50.247669 29861 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:17:50.253317 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.249662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:34027 (local address 127.0.0.1:50882)
0924 07:17:50.249832 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.249870 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:50.249928 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:50.250231 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:50.250234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:50.250236 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:50.250297 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.250299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.251207 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.251208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.252203 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.252206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.253044 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.253045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.253114 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.253129 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 07:17:50.253198 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 07:17:50.253246 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":71,"threads_started":1}
I20260924 07:17:50.253328 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.249758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50882 (local address 127.10.130.194:34027)
0924 07:17:50.249948 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.249953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.249991 (+ 38us) server_negotiation.cc:407] Connection header received
0924 07:17:50.250133 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.250135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.250149 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.250217 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:50.250372 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.251120 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.252293 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.252990 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.253048 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.253153 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.253212 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.253252 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":87,"threads_started":1}
I20260924 07:17:50.253721 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.249662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:37473 (local address 127.0.0.1:51232)
0924 07:17:50.249977 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.249987 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:50.250033 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:50.250484 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:50.250485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:50.250486 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:50.250515 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.250516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.251762 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.251763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.252721 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.252723 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.253503 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.253504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.253561 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.253565 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:17:50.253616 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 07:17:50.253652 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":35,"thread_start_us":67,"threads_started":1}
I20260924 07:17:50.253916 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.249810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.10.130.195:37473)
0924 07:17:50.250293 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.250296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.250304 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:17:50.250340 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.250341 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.250355 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.250420 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:50.250584 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.251655 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.252793 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.253434 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.253504 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.253551 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.253627 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.253839 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":67,"threads_started":1}
I20260924 07:17:50.254078 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.249863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38967 (local address 127.0.0.1:38806)
0924 07:17:50.250023 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.250031 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:50.250066 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:50.250413 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:50.250414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:50.250416 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:50.250465 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.250467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.251770 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.251771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.252889 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.252892 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.253856 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.253857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.253912 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.253917 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 07:17:50.253983 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 07:17:50.254018 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":28,"thread_start_us":74,"threads_started":1}
I20260924 07:17:50.254308 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.250073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38806 (local address 127.10.130.193:38967)
0924 07:17:50.250244 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.250248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.250258 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:17:50.250296 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.250298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.250314 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.250535 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:50.250618 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.251676 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.252971 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.253785 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.253855 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.253905 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.254153 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.254234 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
I20260924 07:17:50.254316 30121 tablet_service.cc:1467] Processing CreateTablet for tablet 2ff3b97951fb443c917a76702a618a5c (DEFAULT_TABLE table=test-table [id=4f08fb323e964bf19bbbc9755cfe6796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:50.254534 30255 tablet_service.cc:1467] Processing CreateTablet for tablet 2ff3b97951fb443c917a76702a618a5c (DEFAULT_TABLE table=test-table [id=4f08fb323e964bf19bbbc9755cfe6796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:50.254638 30121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ff3b97951fb443c917a76702a618a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:50.254802 30255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ff3b97951fb443c917a76702a618a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:50.256723 29987 tablet_service.cc:1467] Processing CreateTablet for tablet 2ff3b97951fb443c917a76702a618a5c (DEFAULT_TABLE table=test-table [id=4f08fb323e964bf19bbbc9755cfe6796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:50.257007 29987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ff3b97951fb443c917a76702a618a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:50.257313 30335 tablet_bootstrap.cc:492] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8: Bootstrap starting.
I20260924 07:17:50.257623 30336 tablet_bootstrap.cc:492] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb: Bootstrap starting.
I20260924 07:17:50.257925 30335 tablet_bootstrap.cc:654] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:50.258224 30336 tablet_bootstrap.cc:654] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:50.258450 30335 log.cc:824] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:50.258747 30336 log.cc:824] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:50.259294 30335 tablet_bootstrap.cc:492] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8: No bootstrap required, opened a new log
I20260924 07:17:50.259354 30335 ts_tablet_manager.cc:1397] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:17:50.259382 30336 tablet_bootstrap.cc:492] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb: No bootstrap required, opened a new log
I20260924 07:17:50.259438 30336 ts_tablet_manager.cc:1397] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:17:50.259871 30339 tablet_bootstrap.cc:492] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344: Bootstrap starting.
I20260924 07:17:50.260504 30339 tablet_bootstrap.cc:654] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:50.260949 30336 raft_consensus.cc:348] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9648a4784a5e4f8797ca367139731feb" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 34027 } } peers { permanent_uuid: "842a057bf30b42108da045df4d37f344" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38967 } } peers { permanent_uuid: "f44e83bf4757469680045bb28f835be8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37473 } }
I20260924 07:17:50.261056 30339 log.cc:824] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:50.261081 30336 raft_consensus.cc:374] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:50.261113 30336 raft_consensus.cc:729] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9648a4784a5e4f8797ca367139731feb, State: Initialized, Role: FOLLOWER
I20260924 07:17:50.261237 30336 consensus_queue.cc:260] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9648a4784a5e4f8797ca367139731feb" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 34027 } } peers { permanent_uuid: "842a057bf30b42108da045df4d37f344" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38967 } } peers { permanent_uuid: "f44e83bf4757469680045bb28f835be8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37473 } }
I20260924 07:17:50.261303 30335 raft_consensus.cc:348] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9648a4784a5e4f8797ca367139731feb" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 34027 } } peers { permanent_uuid: "842a057bf30b42108da045df4d37f344" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38967 } } peers { permanent_uuid: "f44e83bf4757469680045bb28f835be8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37473 } }
I20260924 07:17:50.261461 30335 raft_consensus.cc:374] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:50.261500 30335 raft_consensus.cc:729] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f44e83bf4757469680045bb28f835be8, State: Initialized, Role: FOLLOWER
I20260924 07:17:50.261503 30336 ts_tablet_manager.cc:1428] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:17:50.261538 30187 heartbeater.cc:503] Master 127.10.130.254:39093 was elected leader, sending a full tablet report...
I20260924 07:17:50.261605 30335 consensus_queue.cc:260] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9648a4784a5e4f8797ca367139731feb" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 34027 } } peers { permanent_uuid: "842a057bf30b42108da045df4d37f344" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38967 } } peers { permanent_uuid: "f44e83bf4757469680045bb28f835be8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37473 } }
I20260924 07:17:50.261834 30335 ts_tablet_manager.cc:1428] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:17:50.261884 30321 heartbeater.cc:503] Master 127.10.130.254:39093 was elected leader, sending a full tablet report...
I20260924 07:17:50.262406 30339 tablet_bootstrap.cc:492] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344: No bootstrap required, opened a new log
I20260924 07:17:50.262470 30339 ts_tablet_manager.cc:1397] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260924 07:17:50.263787 30339 raft_consensus.cc:348] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9648a4784a5e4f8797ca367139731feb" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 34027 } } peers { permanent_uuid: "842a057bf30b42108da045df4d37f344" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38967 } } peers { permanent_uuid: "f44e83bf4757469680045bb28f835be8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37473 } }
I20260924 07:17:50.263906 30339 raft_consensus.cc:374] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:50.263935 30339 raft_consensus.cc:729] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842a057bf30b42108da045df4d37f344, State: Initialized, Role: FOLLOWER
I20260924 07:17:50.264063 30339 consensus_queue.cc:260] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9648a4784a5e4f8797ca367139731feb" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 34027 } } peers { permanent_uuid: "842a057bf30b42108da045df4d37f344" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38967 } } peers { permanent_uuid: "f44e83bf4757469680045bb28f835be8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37473 } }
I20260924 07:17:50.264302 30339 ts_tablet_manager.cc:1428] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:17:50.264355 30053 heartbeater.cc:503] Master 127.10.130.254:39093 was elected leader, sending a full tablet report...
I20260924 07:17:50.266186 30343 raft_consensus.cc:482] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:17:50.266249 30343 raft_consensus.cc:504] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9648a4784a5e4f8797ca367139731feb" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 34027 } } peers { permanent_uuid: "842a057bf30b42108da045df4d37f344" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38967 } } peers { permanent_uuid: "f44e83bf4757469680045bb28f835be8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37473 } }
I20260924 07:17:50.266517 30343 leader_election.cc:288] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9648a4784a5e4f8797ca367139731feb (127.10.130.194:34027), f44e83bf4757469680045bb28f835be8 (127.10.130.195:37473)
I20260924 07:17:50.269095 30055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.266633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:34027 (local address 127.10.130.193:52367)
0924 07:17:50.266705 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.266715 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:50.266753 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:50.266906 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:50.266907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:50.266908 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:50.266965 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.266966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.267587 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.267587 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.268361 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.268363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.268873 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.268874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.268945 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.268948 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:17:50.268990 (+ 42us) client_negotiation.cc:240] Negotiation successful
0924 07:17:50.269027 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260924 07:17:50.269095 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.266748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:39111 (local address 127.10.130.195:37473)
0924 07:17:50.266791 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.266793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.266931 (+ 138us) server_negotiation.cc:407] Connection header received
0924 07:17:50.266985 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.266986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.266994 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.267041 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:50.267152 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.267570 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.268487 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.268827 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.268888 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.268895 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.269012 (+ 117us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.269042 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 07:17:50.269101 30331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.266721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:52367 (local address 127.10.130.194:34027)
0924 07:17:50.266753 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.266755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.266762 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:17:50.266802 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.266803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.266811 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.266853 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:50.267019 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.267519 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.268430 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.268809 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.268859 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.268866 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.269003 (+ 137us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.269036 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:17:50.269114 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.266665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:37473 (local address 127.10.130.193:39111)
0924 07:17:50.266874 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.266883 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:50.266941 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:50.267042 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:50.267043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:50.267044 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:50.267100 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.267101 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.267638 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.267639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.268413 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.268415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.268896 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.268897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.268953 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.268957 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:17:50.269002 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 07:17:50.269036 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":67,"threads_started":1}
I20260924 07:17:50.269364 30141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ff3b97951fb443c917a76702a618a5c" candidate_uuid: "842a057bf30b42108da045df4d37f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9648a4784a5e4f8797ca367139731feb" is_pre_election: true
I20260924 07:17:50.269405 30275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ff3b97951fb443c917a76702a618a5c" candidate_uuid: "842a057bf30b42108da045df4d37f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44e83bf4757469680045bb28f835be8" is_pre_election: true
I20260924 07:17:50.269484 30141 raft_consensus.cc:2454] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842a057bf30b42108da045df4d37f344 in term 0.
I20260924 07:17:50.269491 30275 raft_consensus.cc:2454] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 842a057bf30b42108da045df4d37f344 in term 0.
I20260924 07:17:50.269670 29942 leader_election.cc:302] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842a057bf30b42108da045df4d37f344, 9648a4784a5e4f8797ca367139731feb; no voters:
I20260924 07:17:50.269778 30343 raft_consensus.cc:2781] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:17:50.269830 30343 raft_consensus.cc:482] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:17:50.269856 30343 raft_consensus.cc:3037] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:50.270399 30343 raft_consensus.cc:504] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9648a4784a5e4f8797ca367139731feb" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 34027 } } peers { permanent_uuid: "842a057bf30b42108da045df4d37f344" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38967 } } peers { permanent_uuid: "f44e83bf4757469680045bb28f835be8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37473 } }
I20260924 07:17:50.270505 30343 leader_election.cc:288] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [CANDIDATE]: Term 1 election: Requested vote from peers 9648a4784a5e4f8797ca367139731feb (127.10.130.194:34027), f44e83bf4757469680045bb28f835be8 (127.10.130.195:37473)
I20260924 07:17:50.270638 30275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ff3b97951fb443c917a76702a618a5c" candidate_uuid: "842a057bf30b42108da045df4d37f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44e83bf4757469680045bb28f835be8"
I20260924 07:17:50.270701 30275 raft_consensus.cc:3037] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:50.270684 30141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ff3b97951fb443c917a76702a618a5c" candidate_uuid: "842a057bf30b42108da045df4d37f344" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9648a4784a5e4f8797ca367139731feb"
I20260924 07:17:50.270751 30141 raft_consensus.cc:3037] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:50.271303 30275 raft_consensus.cc:2454] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842a057bf30b42108da045df4d37f344 in term 1.
I20260924 07:17:50.271333 30141 raft_consensus.cc:2454] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 842a057bf30b42108da045df4d37f344 in term 1.
I20260924 07:17:50.271449 29941 leader_election.cc:302] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 842a057bf30b42108da045df4d37f344, f44e83bf4757469680045bb28f835be8; no voters:
I20260924 07:17:50.271535 30343 raft_consensus.cc:2781] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:50.271665 30343 raft_consensus.cc:686] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [term 1 LEADER]: Becoming Leader. State: Replica: 842a057bf30b42108da045df4d37f344, State: Running, Role: LEADER
I20260924 07:17:50.271729 30343 consensus_queue.cc:237] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9648a4784a5e4f8797ca367139731feb" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 34027 } } peers { permanent_uuid: "842a057bf30b42108da045df4d37f344" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38967 } } peers { permanent_uuid: "f44e83bf4757469680045bb28f835be8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37473 } }
I20260924 07:17:50.272465 29862 catalog_manager.cc:5495] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 reported cstate change: term changed from 0 to 1, leader changed from <none> to 842a057bf30b42108da045df4d37f344 (127.10.130.193). New cstate: current_term: 1 leader_uuid: "842a057bf30b42108da045df4d37f344" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9648a4784a5e4f8797ca367139731feb" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 34027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "842a057bf30b42108da045df4d37f344" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f44e83bf4757469680045bb28f835be8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37473 } health_report { overall_health: UNKNOWN } } }
I20260924 07:17:50.282009 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.279722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38810 (local address 127.10.130.193:38967)
0924 07:17:50.279772 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.279774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.279782 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:17:50.279837 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.279838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.279857 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.279923 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:17:50.280078 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.280524 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.281258 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.281469 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.281531 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.281709 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:17:50.281802 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:17:50.281872 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.281922 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.281942 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 07:17:50.283542 30141 raft_consensus.cc:1261] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb [term 1 FOLLOWER]: Refusing update from remote peer 842a057bf30b42108da045df4d37f344: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:50.283538 30275 raft_consensus.cc:1261] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8 [term 1 FOLLOWER]: Refusing update from remote peer 842a057bf30b42108da045df4d37f344: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:50.283852 30345 consensus_queue.cc:1035] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9648a4784a5e4f8797ca367139731feb" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 34027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:50.284005 30343 consensus_queue.cc:1035] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f44e83bf4757469680045bb28f835be8" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 37473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:50.286077 30348 mvcc.cc:205] Tried to move back new op lower bound from 7332799571079061504 to 7332799571033563136. Current Snapshot: MvccSnapshot[applied={T|T < 7332799571079061504}]
I20260924 07:17:50.286424 30350 mvcc.cc:205] Tried to move back new op lower bound from 7332799571079061504 to 7332799571033563136. Current Snapshot: MvccSnapshot[applied={T|T < 7332799571079061504}]
I20260924 07:17:50.288496 29862 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:50.288582 29862 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52848:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:50.289319 29862 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27bf8862c1404961a93fcc680eba948e: Sending DeleteTablet for 3 replicas of tablet 2ff3b97951fb443c917a76702a618a5c
I20260924 07:17:50.289623 30255 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ff3b97951fb443c917a76702a618a5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:50 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:51232
I20260924 07:17:50.289673 29987 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ff3b97951fb443c917a76702a618a5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:50 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:38806
I20260924 07:17:50.289718 30121 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ff3b97951fb443c917a76702a618a5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:50 UTC) from {username='kudu', principal='kudu/127.10.130.254@KRBTEST.COM'} at 127.0.0.1:50882
I20260924 07:17:50.289904 30358 tablet_replica.cc:330] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344: stopping tablet replica
I20260924 07:17:50.289971 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 29924
I20260924 07:17:50.289995 30358 raft_consensus.cc:2229] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:17:50.290088 30358 raft_consensus.cc:2258] T 2ff3b97951fb443c917a76702a618a5c P 842a057bf30b42108da045df4d37f344 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:50.290560 30359 tablet_replica.cc:330] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb: stopping tablet replica
I20260924 07:17:50.290623 30359 raft_consensus.cc:2229] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:50.290683 30359 raft_consensus.cc:2258] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:50.291033 30359 ts_tablet_manager.cc:1905] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:50.292425 30359 ts_tablet_manager.cc:1918] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:17:50.292486 30359 log.cc:1196] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/2ff3b97951fb443c917a76702a618a5c
I20260924 07:17:50.292711 30359 ts_tablet_manager.cc:1939] T 2ff3b97951fb443c917a76702a618a5c P 9648a4784a5e4f8797ca367139731feb: Deleting consensus metadata
I20260924 07:17:50.293038 29848 catalog_manager.cc:4841] TS 9648a4784a5e4f8797ca367139731feb (127.10.130.194:34027): tablet 2ff3b97951fb443c917a76702a618a5c (table test-table [id=4f08fb323e964bf19bbbc9755cfe6796]) successfully deleted
W20260924 07:17:50.295271 29848 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:38967 (error 108)
I20260924 07:17:50.295472 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 30058
I20260924 07:17:50.295550 30357 tablet_replica.cc:330] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8: stopping tablet replica
I20260924 07:17:50.295642 30357 raft_consensus.cc:2229] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:50.295730 30357 raft_consensus.cc:2258] T 2ff3b97951fb443c917a76702a618a5c P f44e83bf4757469680045bb28f835be8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:50.296252 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.295627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38967 (local address 127.0.0.1:38816)
0924 07:17:50.296182 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.296222 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:38967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":536}
W20260924 07:17:50.296377 29848 catalog_manager.cc:4584] TS 842a057bf30b42108da045df4d37f344 (127.10.130.193:38967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2ff3b97951fb443c917a76702a618a5c: Network error: Client connection negotiation failed: client connection to 127.10.130.193:38967: connect: Connection refused (error 111)
I20260924 07:17:50.299777 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 30192
I20260924 07:17:50.303604 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 29830
I20260924 07:17:50.303689 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.303583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:37473 (local address 127.0.0.1:51240)
0924 07:17:50.303633 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.303659 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.195:37473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
2026-09-24T07:17:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1728 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-24T07:17:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:17:50Z Disabled control of system clock
I20260924 07:17:50.328431 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39781
--webserver_interface=127.10.130.254
--webserver_port=0
--builtin_ntp_servers=127.10.130.212:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:39781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 07:17:50.404274 30364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:50.404477 30364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:50.404496 30364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:50.405563 30364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:50.405619 30364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:50.405630 30364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:50.405640 30364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:50.405651 30364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:50.406896 30364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:39781
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:39781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:50.407073 30364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:50.407289 30364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:50.411636 30370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:50.411667 30369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:50.411702 30364 server_base.cc:1034] running on GCE node
W20260924 07:17:50.411655 30372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:50.412066 30364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:50.412338 30364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:50.413480 30364 hybrid_clock.cc:648] HybridClock initialized: now 1790234270413464 us; error 27 us; skew 500 ppm
I20260924 07:17:50.414620 30364 webserver.cc:460] Webserver started at http://127.10.130.254:34749/ using document root <none> and password file <none>
I20260924 07:17:50.414827 30364 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:50.414875 30364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:50.414996 30364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:50.416066 30364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "c9e3924956cc48d2ad7c742dbbd00f50"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "341bad9e909ff8a51281dc7ce78a518b"
server_key_iv: "e5cef4b673a5eb8a922a58eb7b0da5df"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.416384 30364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "c9e3924956cc48d2ad7c742dbbd00f50"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "341bad9e909ff8a51281dc7ce78a518b"
server_key_iv: "e5cef4b673a5eb8a922a58eb7b0da5df"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.417615 30364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:17:50.418305 30378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.418457 30364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:50.418537 30364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "c9e3924956cc48d2ad7c742dbbd00f50"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "341bad9e909ff8a51281dc7ce78a518b"
server_key_iv: "e5cef4b673a5eb8a922a58eb7b0da5df"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.418614 30364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:50.437629 30364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:50.437904 30364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:50.438025 30364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:50.441618 30364 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:39781
I20260924 07:17:50.441663 30430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:39781 every 8 connection(s)
I20260924 07:17:50.441962 30364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:50.442862 30431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:50.445212 30431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50: Bootstrap starting.
I20260924 07:17:50.445780 30431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:50.446058 30431 log.cc:824] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:50.446667 30431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50: No bootstrap required, opened a new log
I20260924 07:17:50.448302 30431 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e3924956cc48d2ad7c742dbbd00f50" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39781 } }
I20260924 07:17:50.448437 30431 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:50.448473 30431 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9e3924956cc48d2ad7c742dbbd00f50, State: Initialized, Role: FOLLOWER
I20260924 07:17:50.448613 30431 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e3924956cc48d2ad7c742dbbd00f50" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39781 } }
I20260924 07:17:50.448678 30431 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:50.448701 30431 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:50.448732 30431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:50.449231 30431 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e3924956cc48d2ad7c742dbbd00f50" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39781 } }
I20260924 07:17:50.449322 30431 leader_election.cc:302] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9e3924956cc48d2ad7c742dbbd00f50; no voters:
I20260924 07:17:50.449476 30431 leader_election.cc:288] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:17:50.449519 30434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:50.449671 30434 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [term 1 LEADER]: Becoming Leader. State: Replica: c9e3924956cc48d2ad7c742dbbd00f50, State: Running, Role: LEADER
I20260924 07:17:50.449707 30431 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:50.449765 30434 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e3924956cc48d2ad7c742dbbd00f50" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39781 } }
I20260924 07:17:50.450106 30436 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9e3924956cc48d2ad7c742dbbd00f50. Latest consensus state: current_term: 1 leader_uuid: "c9e3924956cc48d2ad7c742dbbd00f50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e3924956cc48d2ad7c742dbbd00f50" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39781 } } }
I20260924 07:17:50.450186 30436 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:50.450214 30435 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9e3924956cc48d2ad7c742dbbd00f50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9e3924956cc48d2ad7c742dbbd00f50" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 39781 } } }
I20260924 07:17:50.450300 30435 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:50.450395 30441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:50.450879 30441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:50.452047 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 30364
I20260924 07:17:50.452162 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:17:50.452312 30441 catalog_manager.cc:1347] Generated new cluster ID: a40ad75b39364fdf987c61363a74db51
I20260924 07:17:50.452343 10763 external_mini_cluster.cc:1428] Setting key 1e3187b4bab5d28f38abf656cda07ba1
I20260924 07:17:50.452385 30441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:17:50.458150 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.452770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37260 (local address 127.10.130.254:39781)
0924 07:17:50.452922 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.452930 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.453109 (+ 179us) server_negotiation.cc:407] Connection header received
0924 07:17:50.453227 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.453234 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.453300 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.453448 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:50.453615 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.454937 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.455967 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.456585 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.456718 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.457344 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:50.457348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:50.457349 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:50.457559 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:50.457601 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:50.457616 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:50.457686 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.457744 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.457822 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
I20260924 07:17:50.460485 30441 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:17:50.460929 30441 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:50.467885 30441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50: Generated new TSK 0
I20260924 07:17:50.468067 30441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:17:50.470418 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.193:0
--local_ip_for_outbound_sockets=127.10.130.193
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39781
--builtin_ntp_servers=127.10.130.212:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 07:17:50.547297 30455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:50.547474 30455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:50.547493 30455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:50.548614 30455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:50.548650 30455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:50.548673 30455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.193
I20260924 07:17:50.550004 30455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.130.193
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.193
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:50.550215 30455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:50.550472 30455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:50.554998 30460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:50.555039 30461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:50.554987 30463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:50.555346 30455 server_base.cc:1034] running on GCE node
I20260924 07:17:50.555529 30455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:50.555757 30455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:50.556900 30455 hybrid_clock.cc:648] HybridClock initialized: now 1790234270556890 us; error 25 us; skew 500 ppm
I20260924 07:17:50.558045 30455 webserver.cc:460] Webserver started at http://127.10.130.193:33189/ using document root <none> and password file <none>
I20260924 07:17:50.558246 30455 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:50.558292 30455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:50.558411 30455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:50.559381 30455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/data/instance:
uuid: "d56750f2bb884b64bc1488cd259e58d1"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "aff7d3a15f90e3a49b29caea10c3d598"
server_key_iv: "4b627b00acb7ed01daf36ee20b6e81a7"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.559687 30455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance:
uuid: "d56750f2bb884b64bc1488cd259e58d1"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "aff7d3a15f90e3a49b29caea10c3d598"
server_key_iv: "4b627b00acb7ed01daf36ee20b6e81a7"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.560921 30455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:17:50.561602 30469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.561748 30455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:17:50.561823 30455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/wal
uuid: "d56750f2bb884b64bc1488cd259e58d1"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "aff7d3a15f90e3a49b29caea10c3d598"
server_key_iv: "4b627b00acb7ed01daf36ee20b6e81a7"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.561887 30455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:50.589102 30455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:50.589375 30455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:50.589497 30455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:50.589699 30455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:50.589995 30455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:50.590027 30455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.590049 30455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:50.590072 30455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.595196 30455 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.193:38049
I20260924 07:17:50.595250 30582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.193:38049 every 8 connection(s)
I20260924 07:17:50.595523 30455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/data/info.pb
I20260924 07:17:50.600224 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.596185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.193:59239 (local address 127.10.130.254:39781)
0924 07:17:50.596240 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.596242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.596447 (+ 205us) server_negotiation.cc:407] Connection header received
0924 07:17:50.596923 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.596924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.596937 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.596984 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:50.597610 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.598088 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.599004 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.599201 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.599255 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.599955 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:50.599956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:50.599957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:50.599961 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:50.599995 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:50.599999 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:50.600042 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.600121 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.600146 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 07:17:50.600414 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.596156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39781 (local address 127.10.130.193:59239)
0924 07:17:50.596341 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.596373 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:50.596834 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:50.597053 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:50.597056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:50.597066 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:50.597525 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.597530 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.598170 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.598171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.598926 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.598928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.599274 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.599275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.599394 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.599836 (+ 442us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:17:50.599836 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:50.599863 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:17:50.599883 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:50.600043 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:50.600048 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:50.600051 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:17:50.600120 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 07:17:50.600215 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260924 07:17:50.600864 30583 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39781
I20260924 07:17:50.600935 30583 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:50.601063 30583 heartbeater.cc:511] Master 127.10.130.254:39781 requested a full tablet report, sending...
I20260924 07:17:50.601462 30395 ts_manager.cc:194] Registered new tserver with Master: d56750f2bb884b64bc1488cd259e58d1 (127.10.130.193:38049)
I20260924 07:17:50.602044 30395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.130.193:59239
I20260924 07:17:50.604990 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 30455
I20260924 07:17:50.605080 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-0/wal/instance
I20260924 07:17:50.605252 10763 external_mini_cluster.cc:1428] Setting key 85ddf98b75bac98eb103e0c03ae9ffb2
I20260924 07:17:50.606094 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.194:0
--local_ip_for_outbound_sockets=127.10.130.194
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39781
--builtin_ntp_servers=127.10.130.212:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 07:17:50.685319 30586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:50.685520 30586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:50.685552 30586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:50.686662 30586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:50.686698 30586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:50.686730 30586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.194
I20260924 07:17:50.688058 30586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.130.194
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.194
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:50.688279 30586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:50.688555 30586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:50.693112 30591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:50.693121 30592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:50.693121 30594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:50.693202 30586 server_base.cc:1034] running on GCE node
I20260924 07:17:50.693516 30586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:50.693742 30586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:50.694871 30586 hybrid_clock.cc:648] HybridClock initialized: now 1790234270694856 us; error 30 us; skew 500 ppm
I20260924 07:17:50.696034 30586 webserver.cc:460] Webserver started at http://127.10.130.194:37449/ using document root <none> and password file <none>
I20260924 07:17:50.696266 30586 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:50.696311 30586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:50.696435 30586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:50.697499 30586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/data/instance:
uuid: "cf4f066328a7487ba18d0a6cdec481b0"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "6d279b64d3f528f6a621b72d0e61ea54"
server_key_iv: "7cba39466233c8499249cc98927c0af8"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.697803 30586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance:
uuid: "cf4f066328a7487ba18d0a6cdec481b0"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "6d279b64d3f528f6a621b72d0e61ea54"
server_key_iv: "7cba39466233c8499249cc98927c0af8"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.698940 30586 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:17:50.699569 30600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.699759 30586 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:17:50.699836 30586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/wal
uuid: "cf4f066328a7487ba18d0a6cdec481b0"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "6d279b64d3f528f6a621b72d0e61ea54"
server_key_iv: "7cba39466233c8499249cc98927c0af8"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.699901 30586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:50.708786 30586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:50.709013 30586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:50.709131 30586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:50.709311 30586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:50.709579 30586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:50.709612 30586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.709636 30586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:50.709661 30586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.714828 30586 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.194:45805
I20260924 07:17:50.714891 30713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.194:45805 every 8 connection(s)
I20260924 07:17:50.715161 30586 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/data/info.pb
I20260924 07:17:50.719578 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.715786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:57875 (local address 127.10.130.254:39781)
0924 07:17:50.715841 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.715843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.716067 (+ 224us) server_negotiation.cc:407] Connection header received
0924 07:17:50.716529 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.716530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.716539 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.716595 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:50.717186 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.717666 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.718536 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.718707 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.718749 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.719390 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:50.719391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:50.719392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:50.719396 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:50.719411 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:50.719415 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:50.719452 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.719498 (+ 46us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.719522 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 07:17:50.719749 30716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.715753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39781 (local address 127.10.130.194:57875)
0924 07:17:50.715925 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.715963 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:50.716453 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:50.716652 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:50.716656 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:50.716664 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:50.717127 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.717132 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.717724 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.717724 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.718478 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.718480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.718756 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.718756 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.718855 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.719271 (+ 416us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:17:50.719271 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:50.719296 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:17:50.719317 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:50.719454 (+ 137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:50.719456 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:50.719458 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:17:50.719499 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 07:17:50.719568 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260924 07:17:50.720160 30714 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39781
I20260924 07:17:50.720243 30714 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:50.720263 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 30586
I20260924 07:17:50.720369 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/wal/instance
I20260924 07:17:50.720419 30714 heartbeater.cc:511] Master 127.10.130.254:39781 requested a full tablet report, sending...
I20260924 07:17:50.720546 10763 external_mini_cluster.cc:1428] Setting key 470db14ef9df02dc8c0b9d07244bc07e
I20260924 07:17:50.720780 30395 ts_manager.cc:194] Registered new tserver with Master: cf4f066328a7487ba18d0a6cdec481b0 (127.10.130.194:45805)
I20260924 07:17:50.721326 30395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.130.194:57875
I20260924 07:17:50.721539 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.130.195:0
--local_ip_for_outbound_sockets=127.10.130.195
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39781
--builtin_ntp_servers=127.10.130.212:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 07:17:50.798626 30717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:50.798813 30717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:50.798832 30717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:50.799896 30717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:50.799929 30717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:50.799952 30717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.130.195
I20260924 07:17:50.801294 30717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.130.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.130.195
--webserver_port=0
--tserver_master_addrs=127.10.130.254:39781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.130.195
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:50.801504 30717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:50.801786 30717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:50.806341 30725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:50.806349 30722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:50.806384 30723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:50.806555 30717 server_base.cc:1034] running on GCE node
I20260924 07:17:50.806752 30717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:50.806969 30717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:50.808110 30717 hybrid_clock.cc:648] HybridClock initialized: now 1790234270808092 us; error 26 us; skew 500 ppm
I20260924 07:17:50.809265 30717 webserver.cc:460] Webserver started at http://127.10.130.195:35637/ using document root <none> and password file <none>
I20260924 07:17:50.809470 30717 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:50.809516 30717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:50.809638 30717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:50.810624 30717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/data/instance:
uuid: "438715d0416d42158834ac09a929796e"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "0661de91fdab5f0a1a7dc22423165dd7"
server_key_iv: "df059e2f81890dfa78ec4e1fb5dbb543"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.810940 30717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance:
uuid: "438715d0416d42158834ac09a929796e"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "0661de91fdab5f0a1a7dc22423165dd7"
server_key_iv: "df059e2f81890dfa78ec4e1fb5dbb543"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.812119 30717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 07:17:50.812762 30731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.812898 30717 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.812974 30717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/wal
uuid: "438715d0416d42158834ac09a929796e"
format_stamp: "Formatted at 2026-09-24 07:17:50 on dist-test-slave-7q7m"
server_key: "0661de91fdab5f0a1a7dc22423165dd7"
server_key_iv: "df059e2f81890dfa78ec4e1fb5dbb543"
server_key_version: "encryptionkey@0"
I20260924 07:17:50.813040 30717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:50.841598 30717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:50.841898 30717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:50.842031 30717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:50.842257 30717 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 07:17:50.842572 30717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 07:17:50.842608 30717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.842636 30717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 07:17:50.842658 30717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:50.847811 30717 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.195:38207
I20260924 07:17:50.847887 30844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.195:38207 every 8 connection(s)
I20260924 07:17:50.848183 30717 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/data/info.pb
I20260924 07:17:50.853098 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.848828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.195:56145 (local address 127.10.130.254:39781)
0924 07:17:50.848869 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.848871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.849113 (+ 242us) server_negotiation.cc:407] Connection header received
0924 07:17:50.849669 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.849670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.849680 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.849725 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:50.850349 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.850883 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.851822 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.852039 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.852108 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.852873 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:50.852874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:50.852874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:50.852879 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:50.852894 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:50.852898 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:50.852949 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.853010 (+ 61us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.853038 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:17:50.853255 30847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.848824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:39781 (local address 127.10.130.195:56145)
0924 07:17:50.848997 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.849030 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:50.849550 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:50.849787 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:50.849790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:50.849798 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:17:50.850275 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.850279 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.850949 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.850950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.851725 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.851728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.852118 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.852118 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.852223 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.852662 (+ 439us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 07:17:50.852662 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:17:50.852689 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 07:17:50.852710 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:17:50.852954 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:17:50.852957 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:17:50.852959 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 07:17:50.853005 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 07:17:50.853079 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260924 07:17:50.853686 30845 heartbeater.cc:347] Connected to a master server at 127.10.130.254:39781
I20260924 07:17:50.853775 30845 heartbeater.cc:464] Registering TS with master...
I20260924 07:17:50.853943 30845 heartbeater.cc:511] Master 127.10.130.254:39781 requested a full tablet report, sending...
I20260924 07:17:50.854275 30395 ts_manager.cc:194] Registered new tserver with Master: 438715d0416d42158834ac09a929796e (127.10.130.195:38207)
I20260924 07:17:50.854636 30395 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.130.195:56145
I20260924 07:17:50.856276 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 30717
I20260924 07:17:50.856374 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-2/wal/instance
I20260924 07:17:50.856534 10763 external_mini_cluster.cc:1428] Setting key 2c4bf4bbd78175203057e80e093c77fd
I20260924 07:17:50.857285 10763 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 07:17:50.860873 30454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.858455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37266 (local address 127.10.130.254:39781)
0924 07:17:50.858526 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.858527 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.858591 (+ 64us) server_negotiation.cc:407] Connection header received
0924 07:17:50.858626 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.858627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.858636 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.858679 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:50.858789 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.859232 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.859906 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.860138 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.860189 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.860696 (+ 507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:50.860697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:50.860698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:50.860701 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:50.860713 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:50.860717 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:50.860754 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.860785 (+ 31us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.860805 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 07:17:50.862113 30395 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 07:17:50.863544 30395 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 07:17:50.868990 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.865489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38207 (local address 127.0.0.1:56624)
0924 07:17:50.865659 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.865697 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:50.865758 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:50.866096 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:50.866100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:50.866104 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:50.866172 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.866173 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.867114 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.867115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.867955 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.867957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.868718 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.868719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.868796 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.868809 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 07:17:50.868874 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 07:17:50.868923 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":8,"thread_start_us":106,"threads_started":1}
I20260924 07:17:50.868999 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.865799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56624 (local address 127.10.130.195:38207)
0924 07:17:50.865948 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.865952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.865960 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:17:50.865987 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.865989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.866002 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.866082 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:50.866243 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.867043 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.868038 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.868648 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.868709 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.868790 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.868888 (+ 98us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.868920 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260924 07:17:50.869513 30854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.865489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.194:45805 (local address 127.0.0.1:56088)
0924 07:17:50.865737 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.865747 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:50.865797 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:50.866362 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:50.866362 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:50.866364 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:50.866397 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.866398 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.867294 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.867296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.868419 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.868420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.869287 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.869288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.869351 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.869355 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:17:50.869413 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 07:17:50.869449 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
I20260924 07:17:50.869549 30856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.865587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56088 (local address 127.10.130.194:45805)
0924 07:17:50.866202 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.866206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.866214 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:17:50.866250 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.866252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.866266 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.866354 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:50.866471 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.867204 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.868497 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.869212 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.869281 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.869361 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.869428 (+ 67us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.869464 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":88,"threads_started":1}
I20260924 07:17:50.869835 30779 tablet_service.cc:1467] Processing CreateTablet for tablet 9f6a2101b89946ada99d23902f2c3749 (DEFAULT_TABLE table=test-table [id=1aa848bf2e334f0ea4654a1c62284983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:50.870157 30779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f6a2101b89946ada99d23902f2c3749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:50.870299 30853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.865503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38049 (local address 127.0.0.1:59872)
0924 07:17:50.865868 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:50.865875 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:50.865908 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:50.866096 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:50.866100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:50.866102 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:50.866172 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.866174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.868247 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.868248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.869284 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:50.869287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.870077 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.870078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:50.870135 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.870139 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 07:17:50.870201 (+ 62us) client_negotiation.cc:240] Negotiation successful
0924 07:17:50.870238 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260924 07:17:50.870329 30857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:50.865747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59872 (local address 127.10.130.193:38049)
0924 07:17:50.865931 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:50.865935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:50.865945 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:17:50.865985 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:50.865987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:50.866002 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:50.866082 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:50.867123 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.868162 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.869365 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:50.869999 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:50.870069 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:50.870143 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:50.870218 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 07:17:50.870252 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
I20260924 07:17:50.870265 30648 tablet_service.cc:1467] Processing CreateTablet for tablet 9f6a2101b89946ada99d23902f2c3749 (DEFAULT_TABLE table=test-table [id=1aa848bf2e334f0ea4654a1c62284983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:50.870563 30648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f6a2101b89946ada99d23902f2c3749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:50.872763 30859 tablet_bootstrap.cc:492] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e: Bootstrap starting.
I20260924 07:17:50.873063 30860 tablet_bootstrap.cc:492] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0: Bootstrap starting.
I20260924 07:17:50.873477 30859 tablet_bootstrap.cc:654] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:50.873472 30517 tablet_service.cc:1467] Processing CreateTablet for tablet 9f6a2101b89946ada99d23902f2c3749 (DEFAULT_TABLE table=test-table [id=1aa848bf2e334f0ea4654a1c62284983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 07:17:50.873677 30860 tablet_bootstrap.cc:654] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:50.873792 30517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f6a2101b89946ada99d23902f2c3749. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:50.873786 30859 log.cc:824] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:50.873958 30860 log.cc:824] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:50.874432 30859 tablet_bootstrap.cc:492] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e: No bootstrap required, opened a new log
I20260924 07:17:50.874498 30859 ts_tablet_manager.cc:1397] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:17:50.874656 30860 tablet_bootstrap.cc:492] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0: No bootstrap required, opened a new log
I20260924 07:17:50.874716 30860 ts_tablet_manager.cc:1397] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:17:50.876327 30863 tablet_bootstrap.cc:492] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1: Bootstrap starting.
I20260924 07:17:50.876959 30863 tablet_bootstrap.cc:654] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:50.876957 30859 raft_consensus.cc:348] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4f066328a7487ba18d0a6cdec481b0" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 45805 } } peers { permanent_uuid: "438715d0416d42158834ac09a929796e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38207 } } peers { permanent_uuid: "d56750f2bb884b64bc1488cd259e58d1" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38049 } }
I20260924 07:17:50.877050 30860 raft_consensus.cc:348] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4f066328a7487ba18d0a6cdec481b0" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 45805 } } peers { permanent_uuid: "438715d0416d42158834ac09a929796e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38207 } } peers { permanent_uuid: "d56750f2bb884b64bc1488cd259e58d1" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38049 } }
I20260924 07:17:50.877137 30859 raft_consensus.cc:374] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:50.877175 30860 raft_consensus.cc:374] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:50.877173 30859 raft_consensus.cc:729] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 438715d0416d42158834ac09a929796e, State: Initialized, Role: FOLLOWER
I20260924 07:17:50.877204 30860 raft_consensus.cc:729] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf4f066328a7487ba18d0a6cdec481b0, State: Initialized, Role: FOLLOWER
I20260924 07:17:50.877240 30863 log.cc:824] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:50.877300 30860 consensus_queue.cc:260] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4f066328a7487ba18d0a6cdec481b0" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 45805 } } peers { permanent_uuid: "438715d0416d42158834ac09a929796e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38207 } } peers { permanent_uuid: "d56750f2bb884b64bc1488cd259e58d1" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38049 } }
I20260924 07:17:50.877300 30859 consensus_queue.cc:260] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4f066328a7487ba18d0a6cdec481b0" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 45805 } } peers { permanent_uuid: "438715d0416d42158834ac09a929796e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38207 } } peers { permanent_uuid: "d56750f2bb884b64bc1488cd259e58d1" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38049 } }
I20260924 07:17:50.877549 30859 ts_tablet_manager.cc:1428] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:17:50.877548 30860 ts_tablet_manager.cc:1428] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 07:17:50.877660 30845 heartbeater.cc:503] Master 127.10.130.254:39781 was elected leader, sending a full tablet report...
I20260924 07:17:50.877702 30714 heartbeater.cc:503] Master 127.10.130.254:39781 was elected leader, sending a full tablet report...
I20260924 07:17:50.877933 30863 tablet_bootstrap.cc:492] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1: No bootstrap required, opened a new log
I20260924 07:17:50.877986 30863 ts_tablet_manager.cc:1397] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 07:17:50.879777 30863 raft_consensus.cc:348] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4f066328a7487ba18d0a6cdec481b0" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 45805 } } peers { permanent_uuid: "438715d0416d42158834ac09a929796e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38207 } } peers { permanent_uuid: "d56750f2bb884b64bc1488cd259e58d1" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38049 } }
I20260924 07:17:50.879884 30863 raft_consensus.cc:374] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:50.879901 30863 raft_consensus.cc:729] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d56750f2bb884b64bc1488cd259e58d1, State: Initialized, Role: FOLLOWER
I20260924 07:17:50.880002 30863 consensus_queue.cc:260] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4f066328a7487ba18d0a6cdec481b0" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 45805 } } peers { permanent_uuid: "438715d0416d42158834ac09a929796e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38207 } } peers { permanent_uuid: "d56750f2bb884b64bc1488cd259e58d1" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38049 } }
I20260924 07:17:50.880209 30863 ts_tablet_manager.cc:1428] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 07:17:50.880264 30583 heartbeater.cc:503] Master 127.10.130.254:39781 was elected leader, sending a full tablet report...
W20260924 07:17:50.965670 30715 tablet.cc:2369] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 07:17:51.004352 30865 raft_consensus.cc:482] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 07:17:51.004477 30865 raft_consensus.cc:504] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4f066328a7487ba18d0a6cdec481b0" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 45805 } } peers { permanent_uuid: "438715d0416d42158834ac09a929796e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38207 } } peers { permanent_uuid: "d56750f2bb884b64bc1488cd259e58d1" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38049 } }
I20260924 07:17:51.004822 30865 leader_election.cc:288] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 438715d0416d42158834ac09a929796e (127.10.130.195:38207), d56750f2bb884b64bc1488cd259e58d1 (127.10.130.193:38049)
I20260924 07:17:51.007436 30858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:51.005045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:33899 (local address 127.10.130.195:38207)
0924 07:17:51.005101 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:51.005103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:51.005125 (+ 22us) server_negotiation.cc:407] Connection header received
0924 07:17:51.005196 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:51.005197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:51.005206 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:51.005247 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:51.005378 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:51.005855 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:51.006787 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:51.007147 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:51.007205 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:51.007213 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:51.007337 (+ 124us) server_negotiation.cc:299] Negotiation successful
0924 07:17:51.007372 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 07:17:51.007438 30716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:51.004997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38207 (local address 127.10.130.194:33899)
0924 07:17:51.005073 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:51.005085 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:51.005127 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:51.005259 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:51.005259 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:51.005260 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:51.005324 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:51.005326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:51.005929 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:51.005930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:51.006705 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:51.006707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:51.007218 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:51.007218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:51.007280 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:51.007283 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:17:51.007326 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 07:17:51.007365 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":37}
I20260924 07:17:51.007505 30857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:51.005054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.130.194:43943 (local address 127.10.130.193:38049)
0924 07:17:51.005106 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:51.005108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:51.005288 (+ 180us) server_negotiation.cc:407] Connection header received
0924 07:17:51.005338 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:51.005339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:51.005352 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:51.005398 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 07:17:51.005525 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:51.005979 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:51.006859 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:51.007222 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:51.007279 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:51.007286 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:51.007406 (+ 120us) server_negotiation.cc:299] Negotiation successful
0924 07:17:51.007443 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 07:17:51.007537 30868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:51.005003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38049 (local address 127.10.130.194:43943)
0924 07:17:51.005224 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:51.005232 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 07:17:51.005287 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:17:51.005404 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:17:51.005405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:17:51.005406 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 07:17:51.005469 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:51.005470 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:51.006047 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:51.006048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:51.006784 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:17:51.006786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:17:51.007290 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:17:51.007291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:17:51.007351 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:51.007354 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 07:17:51.007396 (+ 42us) client_negotiation.cc:240] Negotiation successful
0924 07:17:51.007429 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":65,"thread_start_us":92,"threads_started":1}
I20260924 07:17:51.007815 30799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6a2101b89946ada99d23902f2c3749" candidate_uuid: "cf4f066328a7487ba18d0a6cdec481b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438715d0416d42158834ac09a929796e" is_pre_election: true
I20260924 07:17:51.007834 30537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6a2101b89946ada99d23902f2c3749" candidate_uuid: "cf4f066328a7487ba18d0a6cdec481b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d56750f2bb884b64bc1488cd259e58d1" is_pre_election: true
I20260924 07:17:51.007966 30537 raft_consensus.cc:2454] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf4f066328a7487ba18d0a6cdec481b0 in term 0.
I20260924 07:17:51.007993 30799 raft_consensus.cc:2454] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf4f066328a7487ba18d0a6cdec481b0 in term 0.
I20260924 07:17:51.008211 30603 leader_election.cc:302] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf4f066328a7487ba18d0a6cdec481b0, d56750f2bb884b64bc1488cd259e58d1; no voters:
I20260924 07:17:51.008335 30865 raft_consensus.cc:2781] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 07:17:51.008392 30865 raft_consensus.cc:482] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 07:17:51.008414 30865 raft_consensus.cc:3037] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:51.009070 30865 raft_consensus.cc:504] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4f066328a7487ba18d0a6cdec481b0" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 45805 } } peers { permanent_uuid: "438715d0416d42158834ac09a929796e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38207 } } peers { permanent_uuid: "d56750f2bb884b64bc1488cd259e58d1" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38049 } }
I20260924 07:17:51.009192 30865 leader_election.cc:288] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [CANDIDATE]: Term 1 election: Requested vote from peers 438715d0416d42158834ac09a929796e (127.10.130.195:38207), d56750f2bb884b64bc1488cd259e58d1 (127.10.130.193:38049)
I20260924 07:17:51.009326 30537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6a2101b89946ada99d23902f2c3749" candidate_uuid: "cf4f066328a7487ba18d0a6cdec481b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d56750f2bb884b64bc1488cd259e58d1"
I20260924 07:17:51.009333 30799 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f6a2101b89946ada99d23902f2c3749" candidate_uuid: "cf4f066328a7487ba18d0a6cdec481b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "438715d0416d42158834ac09a929796e"
I20260924 07:17:51.009399 30799 raft_consensus.cc:3037] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:51.009398 30537 raft_consensus.cc:3037] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:51.010076 30799 raft_consensus.cc:2454] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf4f066328a7487ba18d0a6cdec481b0 in term 1.
I20260924 07:17:51.010161 30537 raft_consensus.cc:2454] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf4f066328a7487ba18d0a6cdec481b0 in term 1.
I20260924 07:17:51.010237 30602 leader_election.cc:302] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 438715d0416d42158834ac09a929796e, cf4f066328a7487ba18d0a6cdec481b0; no voters:
I20260924 07:17:51.010344 30865 raft_consensus.cc:2781] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:51.010505 30865 raft_consensus.cc:686] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [term 1 LEADER]: Becoming Leader. State: Replica: cf4f066328a7487ba18d0a6cdec481b0, State: Running, Role: LEADER
I20260924 07:17:51.010581 30865 consensus_queue.cc:237] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4f066328a7487ba18d0a6cdec481b0" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 45805 } } peers { permanent_uuid: "438715d0416d42158834ac09a929796e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38207 } } peers { permanent_uuid: "d56750f2bb884b64bc1488cd259e58d1" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38049 } }
I20260924 07:17:51.011303 30395 catalog_manager.cc:5495] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf4f066328a7487ba18d0a6cdec481b0 (127.10.130.194). New cstate: current_term: 1 leader_uuid: "cf4f066328a7487ba18d0a6cdec481b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf4f066328a7487ba18d0a6cdec481b0" member_type: VOTER last_known_addr { host: "127.10.130.194" port: 45805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "438715d0416d42158834ac09a929796e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d56750f2bb884b64bc1488cd259e58d1" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38049 } health_report { overall_health: UNKNOWN } } }
I20260924 07:17:51.047554 30856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:51.045434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56096 (local address 127.10.130.194:45805)
0924 07:17:51.045482 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:51.045484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:51.045494 (+ 10us) server_negotiation.cc:407] Connection header received
0924 07:17:51.045533 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:51.045534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:51.045550 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:51.045614 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:17:51.045717 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:51.046178 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:51.046854 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:51.047027 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:51.047068 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:51.047181 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:17:51.047275 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:17:51.047333 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:51.047462 (+ 129us) server_negotiation.cc:299] Negotiation successful
0924 07:17:51.047477 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 07:17:51.049180 30799 raft_consensus.cc:1261] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e [term 1 FOLLOWER]: Refusing update from remote peer cf4f066328a7487ba18d0a6cdec481b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:51.049189 30537 raft_consensus.cc:1261] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1 [term 1 FOLLOWER]: Refusing update from remote peer cf4f066328a7487ba18d0a6cdec481b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:17:51.049510 30869 consensus_queue.cc:1035] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "438715d0416d42158834ac09a929796e" member_type: VOTER last_known_addr { host: "127.10.130.195" port: 38207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:51.049635 30865 consensus_queue.cc:1035] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d56750f2bb884b64bc1488cd259e58d1" member_type: VOTER last_known_addr { host: "127.10.130.193" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:17:51.052156 30873 mvcc.cc:205] Tried to move back new op lower bound from 7332799574215155712 to 7332799574059831296. Current Snapshot: MvccSnapshot[applied={T|T < 7332799574215155712}]
I20260924 07:17:51.052474 30872 mvcc.cc:205] Tried to move back new op lower bound from 7332799574215155712 to 7332799574059831296. Current Snapshot: MvccSnapshot[applied={T|T < 7332799574215155712}]
I20260924 07:17:51.053242 30874 mvcc.cc:205] Tried to move back new op lower bound from 7332799574215155712 to 7332799574059831296. Current Snapshot: MvccSnapshot[applied={T|T < 7332799574215155712 or (T in {7332799574215155712})}]
I20260924 07:17:51.054593 30395 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:51.054682 30395 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 07:17:51.055353 30395 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9e3924956cc48d2ad7c742dbbd00f50: Sending DeleteTablet for 3 replicas of tablet 9f6a2101b89946ada99d23902f2c3749
I20260924 07:17:51.055639 30648 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f6a2101b89946ada99d23902f2c3749 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:51 UTC) from {username='slave'} at 127.0.0.1:56088
I20260924 07:17:51.055662 30517 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f6a2101b89946ada99d23902f2c3749 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:51 UTC) from {username='slave'} at 127.0.0.1:59872
I20260924 07:17:51.055702 30779 tablet_service.cc:1514] Processing DeleteTablet for tablet 9f6a2101b89946ada99d23902f2c3749 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 07:17:51 UTC) from {username='slave'} at 127.0.0.1:56624
I20260924 07:17:51.055876 30883 tablet_replica.cc:330] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1: stopping tablet replica
I20260924 07:17:51.055998 30883 raft_consensus.cc:2229] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:51.056100 30883 raft_consensus.cc:2258] T 9f6a2101b89946ada99d23902f2c3749 P d56750f2bb884b64bc1488cd259e58d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:51.056097 30884 tablet_replica.cc:330] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e: stopping tablet replica
I20260924 07:17:51.056162 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 30455
I20260924 07:17:51.056185 30884 raft_consensus.cc:2229] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 07:17:51.056265 30884 raft_consensus.cc:2258] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:51.056381 30882 tablet_replica.cc:330] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0: stopping tablet replica
I20260924 07:17:51.056449 30882 raft_consensus.cc:2229] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [term 1 LEADER]: Raft consensus shutting down.
I20260924 07:17:51.056536 30882 raft_consensus.cc:2258] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 07:17:51.057219 30882 ts_tablet_manager.cc:1905] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:51.058790 30882 ts_tablet_manager.cc:1918] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 07:17:51.058852 30882 log.cc:1196] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0: Deleting WAL directory at /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790234044967003-10763-0/minicluster-data/ts-1/wal/wals/9f6a2101b89946ada99d23902f2c3749
I20260924 07:17:51.059064 30882 ts_tablet_manager.cc:1939] T 9f6a2101b89946ada99d23902f2c3749 P cf4f066328a7487ba18d0a6cdec481b0: Deleting consensus metadata
I20260924 07:17:51.060851 30382 catalog_manager.cc:4841] TS cf4f066328a7487ba18d0a6cdec481b0 (127.10.130.194:45805): tablet 9f6a2101b89946ada99d23902f2c3749 (table test-table [id=1aa848bf2e334f0ea4654a1c62284983]) successfully deleted
W20260924 07:17:51.061012 30381 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.130.193:38049 (error 108)
I20260924 07:17:51.061334 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 30586
I20260924 07:17:51.061910 30853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:51.061400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.193:38049 (local address 127.0.0.1:59880)
0924 07:17:51.061835 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:51.061877 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.193:38049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413}
W20260924 07:17:51.062016 30381 catalog_manager.cc:4584] TS d56750f2bb884b64bc1488cd259e58d1 (127.10.130.193:38049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9f6a2101b89946ada99d23902f2c3749: Network error: Client connection negotiation failed: client connection to 127.10.130.193:38049: connect: Connection refused (error 111)
I20260924 07:17:51.064785 30884 ts_tablet_manager.cc:1905] T 9f6a2101b89946ada99d23902f2c3749 P 438715d0416d42158834ac09a929796e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 07:17:51.066152 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 30717
I20260924 07:17:51.070381 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 30364
I20260924 07:17:51.070470 30853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:51.070375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.195:38207 (local address 127.0.0.1:56634)
0924 07:17:51.070417 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0924 07:17:51.070443 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.195:38207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
2026-09-24T07:17:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (766 ms)
[----------] 6 tests from EncryptionPolicyTest (7647 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:17:51 dist-test-slave-7q7m krb5kdc[30888](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:17:51 dist-test-slave-7q7m krb5kdc[30888](info): set up 2 sockets
Sep 24 07:17:51 dist-test-slave-7q7m krb5kdc[30888](info): commencing operation
krb5kdc: starting...
W20260924 07:17:52.011703 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.923s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:17:52 dist-test-slave-7q7m krb5kdc[30888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:17:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:17:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 07:17:52.116910 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41207
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:52.195587 30901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:52.195780 30901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:52.195811 30901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:52.196940 30901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:52.196980 30901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:52.197000 30901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:52.197016 30901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:52.197031 30901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:52.198426 30901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41207
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:52.198662 30901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:52.198854 30901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:52.203229 30909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:52.203238 30906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:52.203254 30901 server_base.cc:1034] running on GCE node
W20260924 07:17:52.203227 30907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:52.203646 30901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:52.203893 30901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:52.205026 30901 hybrid_clock.cc:648] HybridClock initialized: now 1790234272205007 us; error 32 us; skew 500 ppm
Sep 24 07:17:52 dist-test-slave-7q7m krb5kdc[30888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234272, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:52.207216 30901 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 07:17:52.207590 30901 webserver.cc:460] Webserver started at http://127.0.0.1:38671/ using document root <none> and password file <none>
I20260924 07:17:52.207767 30901 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:52.207813 30901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:52.208029 30901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:52.208997 30901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "43dd730c8e484091a207ba104e7dd0cf"
format_stamp: "Formatted at 2026-09-24 07:17:52 on dist-test-slave-7q7m"
server_key: "48693d766d3428054d66014ad8bbafda"
server_key_iv: "a3faeaaafdc8af1c27a580127d5c08fb"
server_key_version: "encryptionkey@0"
I20260924 07:17:52.209318 30901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "43dd730c8e484091a207ba104e7dd0cf"
format_stamp: "Formatted at 2026-09-24 07:17:52 on dist-test-slave-7q7m"
server_key: "48693d766d3428054d66014ad8bbafda"
server_key_iv: "a3faeaaafdc8af1c27a580127d5c08fb"
server_key_version: "encryptionkey@0"
I20260924 07:17:52.210505 30901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:17:52.211102 30916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:52.211251 30901 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:17:52.211325 30901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "43dd730c8e484091a207ba104e7dd0cf"
format_stamp: "Formatted at 2026-09-24 07:17:52 on dist-test-slave-7q7m"
server_key: "48693d766d3428054d66014ad8bbafda"
server_key_iv: "a3faeaaafdc8af1c27a580127d5c08fb"
server_key_version: "encryptionkey@0"
I20260924 07:17:52.211388 30901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:52.226874 30901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:52.228439 30901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:52.228556 30901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:52.231935 30901 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41207
I20260924 07:17:52.232019 30968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41207 every 8 connection(s)
I20260924 07:17:52.232314 30901 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:52.233053 30969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:52.235328 30969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf: Bootstrap starting.
I20260924 07:17:52.235873 30969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:52.236155 30969 log.cc:824] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:52.236788 30969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf: No bootstrap required, opened a new log
I20260924 07:17:52.238276 30969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43dd730c8e484091a207ba104e7dd0cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } }
I20260924 07:17:52.238400 30969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:52.238431 30969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43dd730c8e484091a207ba104e7dd0cf, State: Initialized, Role: FOLLOWER
I20260924 07:17:52.238564 30969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43dd730c8e484091a207ba104e7dd0cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } }
I20260924 07:17:52.238636 30969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:52.238664 30969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:52.238709 30969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:52.239172 30969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43dd730c8e484091a207ba104e7dd0cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } }
I20260924 07:17:52.239261 30969 leader_election.cc:302] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43dd730c8e484091a207ba104e7dd0cf; no voters:
I20260924 07:17:52.239399 30969 leader_election.cc:288] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:17:52.239434 30972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:52.239619 30972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [term 1 LEADER]: Becoming Leader. State: Replica: 43dd730c8e484091a207ba104e7dd0cf, State: Running, Role: LEADER
I20260924 07:17:52.239621 30969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:52.239719 30972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43dd730c8e484091a207ba104e7dd0cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } }
I20260924 07:17:52.240110 30973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43dd730c8e484091a207ba104e7dd0cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43dd730c8e484091a207ba104e7dd0cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } } }
I20260924 07:17:52.240203 30973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:52.240217 30974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43dd730c8e484091a207ba104e7dd0cf. Latest consensus state: current_term: 1 leader_uuid: "43dd730c8e484091a207ba104e7dd0cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43dd730c8e484091a207ba104e7dd0cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41207 } } }
I20260924 07:17:52.240266 30974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:52.240420 30981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:52.240911 30981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:52.241189 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 30901
I20260924 07:17:52.241302 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:17:52.241488 10763 external_mini_cluster.cc:1428] Setting key 6243175c471e022f674c2b60f29185f0
I20260924 07:17:52.242884 30981 catalog_manager.cc:1347] Generated new cluster ID: a236f788374e4f608badc8b00344df86
I20260924 07:17:52.242928 30981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:17:52.247493 30981 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:17:52.248015 30981 catalog_manager.cc:1504] Loading token signing keys...
Sep 24 07:17:52 dist-test-slave-7q7m krb5kdc[30888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 07:17:52.252785 30981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43dd730c8e484091a207ba104e7dd0cf: Generated new TSK 0
I20260924 07:17:52.252923 30981 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:17:52.254433 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:52.241924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51477 (local address 127.0.0.1:41207)
0924 07:17:52.244133 (+ 2209us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:52.244141 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:52.244170 (+ 29us) server_negotiation.cc:407] Connection header received
0924 07:17:52.244272 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:52.244279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:52.244334 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:52.244457 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:52.245164 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:52.246875 (+ 1711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:52.247927 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:52.248513 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:52.248618 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:52.251173 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:52.251177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:52.251177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:52.251188 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:52.253019 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:52.253367 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:52.253370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:52.253370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:52.253471 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:52.253641 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:52.253642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:52.253642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:52.253890 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:52.254026 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:52.254123 (+ 97us) server_negotiation.cc:299] Negotiation successful
0924 07:17:52.254218 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2173,"thread_start_us":71,"threads_started":1}
I20260924 07:17:52.255261 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:17:52.262529 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:52.256614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42173 (local address 127.0.0.1:41207)
0924 07:17:52.256649 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:52.256650 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:52.256749 (+ 99us) server_negotiation.cc:407] Connection header received
0924 07:17:52.256809 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:52.256810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:52.256819 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:52.256856 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:52.257420 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:52.257895 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:52.258903 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:52.259076 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:52.259116 (+ 40us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:52.260348 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:52.260349 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:52.260349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:52.260354 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:52.261514 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:52.261898 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:52.261898 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:52.261899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:52.261947 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:52.262144 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:52.262145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:52.262145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:52.262228 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:52.262292 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:52.262428 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 07:17:52.262472 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:17:52.264312 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 30901
W20260924 07:17:52.267880 30886 connection.cc:535] client connection to 127.0.0.1:41207 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41207): Connection reset by peer (error 104)
2026-09-24T07:17:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1192 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:17:52 dist-test-slave-7q7m krb5kdc[31000](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:17:52 dist-test-slave-7q7m krb5kdc[31000](info): set up 2 sockets
Sep 24 07:17:52 dist-test-slave-7q7m krb5kdc[31000](info): commencing operation
krb5kdc: starting...
W20260924 07:17:53.199658 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:17:53 dist-test-slave-7q7m krb5kdc[31000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234273, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:17:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:17:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 07:17:53.306515 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46875
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:53.382510 31013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:53.382705 31013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:53.382735 31013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:53.383817 31013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:53.383855 31013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:53.383873 31013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:53.383888 31013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:53.383944 31013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:53.385365 31013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46875
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:53.385589 31013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:53.385771 31013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:53.390065 31018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:53.390065 31021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:53.390105 31019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:53.390223 31013 server_base.cc:1034] running on GCE node
I20260924 07:17:53.390528 31013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:53.390791 31013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:53.391930 31013 hybrid_clock.cc:648] HybridClock initialized: now 1790234273391911 us; error 30 us; skew 500 ppm
Sep 24 07:17:53 dist-test-slave-7q7m krb5kdc[31000](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234273, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:53.394134 31013 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 07:17:53.394505 31013 webserver.cc:460] Webserver started at http://127.0.0.1:43571/ using document root <none> and password file <none>
I20260924 07:17:53.394686 31013 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:53.394731 31013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:53.394848 31013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:53.395814 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "ca69fb7da856405a96e39719d90db4f2"
format_stamp: "Formatted at 2026-09-24 07:17:53 on dist-test-slave-7q7m"
server_key: "325bfb5cc78df13e94303abdeb9f1694"
server_key_iv: "11d5edd7cc2aafb60029fc1e577fbc8d"
server_key_version: "encryptionkey@0"
I20260924 07:17:53.396157 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "ca69fb7da856405a96e39719d90db4f2"
format_stamp: "Formatted at 2026-09-24 07:17:53 on dist-test-slave-7q7m"
server_key: "325bfb5cc78df13e94303abdeb9f1694"
server_key_iv: "11d5edd7cc2aafb60029fc1e577fbc8d"
server_key_version: "encryptionkey@0"
I20260924 07:17:53.397326 31013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 07:17:53.397943 31028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:53.398082 31013 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:17:53.398161 31013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "ca69fb7da856405a96e39719d90db4f2"
format_stamp: "Formatted at 2026-09-24 07:17:53 on dist-test-slave-7q7m"
server_key: "325bfb5cc78df13e94303abdeb9f1694"
server_key_iv: "11d5edd7cc2aafb60029fc1e577fbc8d"
server_key_version: "encryptionkey@0"
I20260924 07:17:53.398219 31013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:53.421532 31013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:53.423166 31013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:53.423295 31013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:53.426885 31013 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46875
I20260924 07:17:53.426918 31080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46875 every 8 connection(s)
I20260924 07:17:53.427228 31013 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:53.428053 31081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:53.430208 31081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2: Bootstrap starting.
I20260924 07:17:53.430517 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 31013
I20260924 07:17:53.430639 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:17:53.430811 10763 external_mini_cluster.cc:1428] Setting key 1871d176eda7db14be1a1097c1b53cbe
I20260924 07:17:53.430850 31081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:53.431115 31081 log.cc:824] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:53.431807 31081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2: No bootstrap required, opened a new log
I20260924 07:17:53.433530 31081 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca69fb7da856405a96e39719d90db4f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260924 07:17:53.433665 31081 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:53.433696 31081 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca69fb7da856405a96e39719d90db4f2, State: Initialized, Role: FOLLOWER
I20260924 07:17:53.433815 31081 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca69fb7da856405a96e39719d90db4f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260924 07:17:53.433878 31081 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:53.433899 31081 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:53.433941 31081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:53.434430 31081 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca69fb7da856405a96e39719d90db4f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260924 07:17:53.434521 31081 leader_election.cc:302] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca69fb7da856405a96e39719d90db4f2; no voters:
I20260924 07:17:53.434679 31081 leader_election.cc:288] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:17:53.434710 31086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:53.434819 31086 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [term 1 LEADER]: Becoming Leader. State: Replica: ca69fb7da856405a96e39719d90db4f2, State: Running, Role: LEADER
I20260924 07:17:53.434918 31081 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:53.435045 31086 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca69fb7da856405a96e39719d90db4f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } }
I20260924 07:17:53.435441 31087 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca69fb7da856405a96e39719d90db4f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca69fb7da856405a96e39719d90db4f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } } }
I20260924 07:17:53.435516 31087 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:53.435685 31088 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca69fb7da856405a96e39719d90db4f2. Latest consensus state: current_term: 1 leader_uuid: "ca69fb7da856405a96e39719d90db4f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca69fb7da856405a96e39719d90db4f2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46875 } } }
I20260924 07:17:53.435705 31094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:53.435751 31088 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:53.436611 31094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:53.437893 31094 catalog_manager.cc:1347] Generated new cluster ID: 1082355d27a74ac0bb05f567939db05b
I20260924 07:17:53.437942 31094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:17:53.440171 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:53.431272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52341 (local address 127.0.0.1:46875)
0924 07:17:53.431427 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:53.431434 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:53.431464 (+ 30us) server_negotiation.cc:407] Connection header received
0924 07:17:53.431877 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:53.431885 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:53.431944 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:53.432096 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:53.432290 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:53.433141 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:53.433995 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:53.434389 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:53.434482 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:53.435166 (+ 684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:53.435169 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:53.435170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:53.435361 (+ 191us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:53.435914 (+ 553us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:53.435931 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:53.436051 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:53.436167 (+ 116us) server_negotiation.cc:299] Negotiation successful
0924 07:17:53.436291 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260924 07:17:53.441072 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:17:53.445683 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:53.442370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46425 (local address 127.0.0.1:46875)
0924 07:17:53.442401 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:53.442402 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:53.442550 (+ 148us) server_negotiation.cc:407] Connection header received
0924 07:17:53.442608 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:53.442609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:53.442619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:53.442680 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:53.442776 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:53.443481 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:53.444468 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:53.444793 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:53.444860 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:53.445462 (+ 602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:53.445463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:53.445463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:53.445467 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:53.445481 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:53.445485 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:53.445543 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:53.445603 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 07:17:53.445612 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:17:53.447537 31094 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:17:53.448217 31094 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:53.456532 31094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca69fb7da856405a96e39719d90db4f2: Generated new TSK 0
I20260924 07:17:53.456728 31094 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:17:53.460431 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 31013
W20260924 07:17:53.463939 30998 connection.cc:535] client connection to 127.0.0.1:46875 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46875): Connection reset by peer (error 104)
2026-09-24T07:17:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1196 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:17:53 dist-test-slave-7q7m krb5kdc[31112](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:17:53 dist-test-slave-7q7m krb5kdc[31112](info): set up 2 sockets
Sep 24 07:17:53 dist-test-slave-7q7m krb5kdc[31112](info): commencing operation
krb5kdc: starting...
W20260924 07:17:54.395627 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:17:54 dist-test-slave-7q7m krb5kdc[31112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:17:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:17:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 07:17:54.505803 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:54.582245 31125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:54.582434 31125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:54.582463 31125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:54.583561 31125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:54.583606 31125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:54.583623 31125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:54.583638 31125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:54.583653 31125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:54.585071 31125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:54.585300 31125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:54.585491 31125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:54.590018 31133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:54.590051 31125 server_base.cc:1034] running on GCE node
W20260924 07:17:54.590037 31131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:54.590023 31130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:54.590507 31125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:54.590777 31125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:54.591917 31125 hybrid_clock.cc:648] HybridClock initialized: now 1790234274591904 us; error 30 us; skew 500 ppm
Sep 24 07:17:54 dist-test-slave-7q7m krb5kdc[31112](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234274, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:54.594203 31125 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 07:17:54.594605 31125 webserver.cc:460] Webserver started at http://127.0.0.1:45641/ using document root <none> and password file <none>
I20260924 07:17:54.594797 31125 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:54.594843 31125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:54.594964 31125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:54.596009 31125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "35d0fba81f4540a0aff10053a5b54631"
format_stamp: "Formatted at 2026-09-24 07:17:54 on dist-test-slave-7q7m"
server_key: "5fcb68f05b1ef0cb4ccdf3657175948f"
server_key_iv: "83e86021815578dd2c7d9f3a2be977fb"
server_key_version: "encryptionkey@0"
I20260924 07:17:54.596360 31125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "35d0fba81f4540a0aff10053a5b54631"
format_stamp: "Formatted at 2026-09-24 07:17:54 on dist-test-slave-7q7m"
server_key: "5fcb68f05b1ef0cb4ccdf3657175948f"
server_key_iv: "83e86021815578dd2c7d9f3a2be977fb"
server_key_version: "encryptionkey@0"
I20260924 07:17:54.597644 31125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 07:17:54.598302 31140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:54.598443 31125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:17:54.598522 31125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "35d0fba81f4540a0aff10053a5b54631"
format_stamp: "Formatted at 2026-09-24 07:17:54 on dist-test-slave-7q7m"
server_key: "5fcb68f05b1ef0cb4ccdf3657175948f"
server_key_iv: "83e86021815578dd2c7d9f3a2be977fb"
server_key_version: "encryptionkey@0"
I20260924 07:17:54.598587 31125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:54.601742 31125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:54.603355 31125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:54.603475 31125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:54.606945 31125 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36225
I20260924 07:17:54.606984 31192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36225 every 8 connection(s)
I20260924 07:17:54.607255 31125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:54.608150 31193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:54.609678 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 31125
I20260924 07:17:54.609792 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:17:54.609962 10763 external_mini_cluster.cc:1428] Setting key 75e142da7134dae166e7d94f5b5fbea5
I20260924 07:17:54.610496 31193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631: Bootstrap starting.
I20260924 07:17:54.611014 31193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:54.611265 31193 log.cc:824] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:54.612012 31193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631: No bootstrap required, opened a new log
I20260924 07:17:54.612202 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:54.610413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39999 (local address 127.0.0.1:36225)
0924 07:17:54.610562 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:54.610571 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:54.610603 (+ 32us) server_negotiation.cc:407] Connection header received
0924 07:17:54.611005 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:54.611011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:54.611035 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:54.611183 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:54.611735 (+ 552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:54.611738 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:54.611739 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:54.611743 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:54.611786 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:54.611800 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:54.611885 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:54.611966 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 07:17:54.612024 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260924 07:17:54.613056 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:17:54.614116 31193 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d0fba81f4540a0aff10053a5b54631" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36225 } }
I20260924 07:17:54.614282 31193 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:54.614326 31193 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35d0fba81f4540a0aff10053a5b54631, State: Initialized, Role: FOLLOWER
I20260924 07:17:54.614439 31193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d0fba81f4540a0aff10053a5b54631" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36225 } }
I20260924 07:17:54.614535 31193 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:54.614574 31193 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:54.614619 31193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:54.615041 31193 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d0fba81f4540a0aff10053a5b54631" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36225 } }
I20260924 07:17:54.615127 31193 leader_election.cc:302] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35d0fba81f4540a0aff10053a5b54631; no voters:
I20260924 07:17:54.615232 31193 leader_election.cc:288] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:17:54.615264 31203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:54.615422 31193 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:54.615422 31203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [term 1 LEADER]: Becoming Leader. State: Replica: 35d0fba81f4540a0aff10053a5b54631, State: Running, Role: LEADER
I20260924 07:17:54.615604 31203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d0fba81f4540a0aff10053a5b54631" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36225 } }
I20260924 07:17:54.615901 31204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35d0fba81f4540a0aff10053a5b54631" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d0fba81f4540a0aff10053a5b54631" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36225 } } }
I20260924 07:17:54.616019 31204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:54.615907 31205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35d0fba81f4540a0aff10053a5b54631. Latest consensus state: current_term: 1 leader_uuid: "35d0fba81f4540a0aff10053a5b54631" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35d0fba81f4540a0aff10053a5b54631" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36225 } } }
I20260924 07:17:54.616146 31205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631 [sys.catalog]: This master's current role is: LEADER
W20260924 07:17:54.617022 31219 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:17:54.617074 31219 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:17:54.617162 31212 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:54.617347 31212 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:54.617774 31196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:54.614491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36273 (local address 127.0.0.1:36225)
0924 07:17:54.614520 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:54.614522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:54.616854 (+ 2332us) server_negotiation.cc:407] Connection header received
0924 07:17:54.616924 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:54.616925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:54.616931 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:54.616974 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:54.617557 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:54.617558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:54.617559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:54.617562 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:54.617577 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:54.617582 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:54.617638 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:54.617705 (+ 67us) server_negotiation.cc:299] Negotiation successful
0924 07:17:54.617716 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260924 07:17:54.618685 31212 catalog_manager.cc:1347] Generated new cluster ID: 75feae45f49c4a4fadb5a37b6cfb59f0
I20260924 07:17:54.618734 31212 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:17:54.631352 31212 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:17:54.631500 31212 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:54.635326 31212 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35d0fba81f4540a0aff10053a5b54631: Generated new TSK 0
I20260924 07:17:54.635473 31212 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:17:54.636925 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 31125
2026-09-24T07:17:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1176 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:17:54 dist-test-slave-7q7m krb5kdc[31224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:17:54 dist-test-slave-7q7m krb5kdc[31224](info): set up 2 sockets
Sep 24 07:17:54 dist-test-slave-7q7m krb5kdc[31224](info): commencing operation
krb5kdc: starting...
W20260924 07:17:55.575663 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:17:55 dist-test-slave-7q7m krb5kdc[31224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:17:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:17:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 07:17:55.678529 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33691
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:55.754843 31237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:55.755029 31237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:55.755050 31237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:55.756214 31237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:55.756256 31237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:55.756269 31237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:55.756282 31237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:55.756304 31237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:55.757699 31237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:55.757907 31237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:55.758102 31237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:55.762462 31245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:55.762560 31237 server_base.cc:1034] running on GCE node
W20260924 07:17:55.762435 31243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:55.762455 31242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:55.762887 31237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:55.763144 31237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:55.764269 31237 hybrid_clock.cc:648] HybridClock initialized: now 1790234275764248 us; error 32 us; skew 500 ppm
Sep 24 07:17:55 dist-test-slave-7q7m krb5kdc[31224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234275, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:55.769564 31237 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 07:17:55.769995 31237 webserver.cc:460] Webserver started at http://127.0.0.1:43321/ using document root <none> and password file <none>
I20260924 07:17:55.770192 31237 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:55.770241 31237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:55.770359 31237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:55.771364 31237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "855efc1aa24f4f3eaba44b8c8dac8947"
format_stamp: "Formatted at 2026-09-24 07:17:55 on dist-test-slave-7q7m"
server_key: "6614798a810cea4cad5c53132829bfd0"
server_key_iv: "471ea9fc8a19d9fcf0538c37be42fab9"
server_key_version: "encryptionkey@0"
I20260924 07:17:55.771677 31237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "855efc1aa24f4f3eaba44b8c8dac8947"
format_stamp: "Formatted at 2026-09-24 07:17:55 on dist-test-slave-7q7m"
server_key: "6614798a810cea4cad5c53132829bfd0"
server_key_iv: "471ea9fc8a19d9fcf0538c37be42fab9"
server_key_version: "encryptionkey@0"
I20260924 07:17:55.772913 31237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:17:55.773497 31252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:55.773625 31237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:17:55.773703 31237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "855efc1aa24f4f3eaba44b8c8dac8947"
format_stamp: "Formatted at 2026-09-24 07:17:55 on dist-test-slave-7q7m"
server_key: "6614798a810cea4cad5c53132829bfd0"
server_key_iv: "471ea9fc8a19d9fcf0538c37be42fab9"
server_key_version: "encryptionkey@0"
I20260924 07:17:55.773757 31237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:55.804844 31237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:55.806744 31237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:55.806871 31237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:55.810530 31237 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33691
I20260924 07:17:55.810560 31304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33691 every 8 connection(s)
I20260924 07:17:55.810873 31237 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:55.811663 31305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:55.812351 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 31237
I20260924 07:17:55.812455 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:17:55.812605 10763 external_mini_cluster.cc:1428] Setting key 4c3e53a0ab26c06687767939020395fa
I20260924 07:17:55.814494 31305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947: Bootstrap starting.
I20260924 07:17:55.815163 31305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:55.815461 31305 log.cc:824] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:55.816143 31305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947: No bootstrap required, opened a new log
I20260924 07:17:55.818145 31305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855efc1aa24f4f3eaba44b8c8dac8947" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33691 } }
I20260924 07:17:55.818274 31305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:55.818295 31305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 855efc1aa24f4f3eaba44b8c8dac8947, State: Initialized, Role: FOLLOWER
I20260924 07:17:55.818396 31305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855efc1aa24f4f3eaba44b8c8dac8947" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33691 } }
I20260924 07:17:55.818459 31305 raft_consensus.cc:388] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:55.818480 31305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:55.818521 31305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [term 0 FOLLOWER]: Advancing to term 1
Sep 24 07:17:55 dist-test-slave-7q7m krb5kdc[31224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 07:17:55.819162 31305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855efc1aa24f4f3eaba44b8c8dac8947" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33691 } }
I20260924 07:17:55.819258 31305 leader_election.cc:302] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 855efc1aa24f4f3eaba44b8c8dac8947; no voters:
I20260924 07:17:55.819411 31305 leader_election.cc:288] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:17:55.819453 31310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:55.819665 31310 raft_consensus.cc:686] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [term 1 LEADER]: Becoming Leader. State: Replica: 855efc1aa24f4f3eaba44b8c8dac8947, State: Running, Role: LEADER
I20260924 07:17:55.819768 31310 consensus_queue.cc:237] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855efc1aa24f4f3eaba44b8c8dac8947" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33691 } }
I20260924 07:17:55.819670 31305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:55.820303 31311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "855efc1aa24f4f3eaba44b8c8dac8947" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855efc1aa24f4f3eaba44b8c8dac8947" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33691 } } }
I20260924 07:17:55.820380 31311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:55.820770 31312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 855efc1aa24f4f3eaba44b8c8dac8947. Latest consensus state: current_term: 1 leader_uuid: "855efc1aa24f4f3eaba44b8c8dac8947" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855efc1aa24f4f3eaba44b8c8dac8947" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33691 } } }
I20260924 07:17:55.820869 31312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:55.820996 31315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:55.821487 31315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:55.822988 31315 catalog_manager.cc:1347] Generated new cluster ID: cc44188b61f340d4ae8ee5ed501e054d
I20260924 07:17:55.823036 31315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:17:55.823503 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:55.813067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54625 (local address 127.0.0.1:33691)
0924 07:17:55.813244 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:55.813253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:55.813285 (+ 32us) server_negotiation.cc:407] Connection header received
0924 07:17:55.813783 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:55.813792 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:55.813856 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:55.813995 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:55.814565 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:55.815844 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:55.816651 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:55.817263 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:55.817375 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:55.819725 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:55.819729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:55.819730 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:55.819741 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:55.821711 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:55.822081 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:55.822084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:55.822084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:55.822189 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:55.822382 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:55.822383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:55.822383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:55.822658 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:55.822789 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:55.823206 (+ 417us) server_negotiation.cc:299] Negotiation successful
0924 07:17:55.823324 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260924 07:17:55.824822 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:17:55.834686 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:55.828231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36367 (local address 127.0.0.1:33691)
0924 07:17:55.828267 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:55.828268 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:55.828348 (+ 80us) server_negotiation.cc:407] Connection header received
0924 07:17:55.828407 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:55.828408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:55.828416 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:55.828467 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:55.829052 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:55.829747 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:55.830733 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:55.831025 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:55.831081 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:55.832354 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:55.832356 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:55.832356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:55.832362 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:55.833727 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:55.834067 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:55.834067 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:55.834068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:55.834113 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:55.834272 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:55.834272 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:55.834272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:55.834372 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:55.834465 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:55.834545 (+ 80us) server_negotiation.cc:299] Negotiation successful
0924 07:17:55.834607 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:17:55.838240 31315 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:17:55.838913 31315 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:55.847247 31315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 855efc1aa24f4f3eaba44b8c8dac8947: Generated new TSK 0
I20260924 07:17:55.847426 31315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:17:55.849745 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 31237
2026-09-24T07:17:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1212 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:17:55 dist-test-slave-7q7m krb5kdc[31336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:17:55 dist-test-slave-7q7m krb5kdc[31336](info): set up 2 sockets
Sep 24 07:17:55 dist-test-slave-7q7m krb5kdc[31336](info): commencing operation
krb5kdc: starting...
W20260924 07:17:56.787544 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.921s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:17:56 dist-test-slave-7q7m krb5kdc[31336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:17:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:17:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 07:17:56.892285 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:56.968839 31349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:56.969028 31349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:56.969058 31349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:56.970155 31349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:56.970191 31349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:56.970209 31349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:56.970223 31349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:56.970239 31349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:56.971639 31349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:56.971868 31349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:56.972090 31349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 07:17:56.976419 31349 server_base.cc:1034] running on GCE node
W20260924 07:17:56.976390 31354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:56.976398 31357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:56.976372 31355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:56.976812 31349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:56.977082 31349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:56.978214 31349 hybrid_clock.cc:648] HybridClock initialized: now 1790234276978202 us; error 29 us; skew 500 ppm
Sep 24 07:17:56 dist-test-slave-7q7m krb5kdc[31336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234276, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:56.980499 31349 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 07:17:56.980894 31349 webserver.cc:460] Webserver started at http://127.0.0.1:38069/ using document root <none> and password file <none>
I20260924 07:17:56.981091 31349 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:56.981137 31349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:56.981251 31349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:56.982214 31349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "330ddff71c974d74a26ec2375657e004"
format_stamp: "Formatted at 2026-09-24 07:17:56 on dist-test-slave-7q7m"
server_key: "6da9553bc7e274d6fc34b1cf2ec417f7"
server_key_iv: "3ead3adcc8f43c67ea144687b3b76874"
server_key_version: "encryptionkey@0"
I20260924 07:17:56.982522 31349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "330ddff71c974d74a26ec2375657e004"
format_stamp: "Formatted at 2026-09-24 07:17:56 on dist-test-slave-7q7m"
server_key: "6da9553bc7e274d6fc34b1cf2ec417f7"
server_key_iv: "3ead3adcc8f43c67ea144687b3b76874"
server_key_version: "encryptionkey@0"
I20260924 07:17:56.983702 31349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:17:56.984385 31364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:56.984529 31349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:56.984606 31349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "330ddff71c974d74a26ec2375657e004"
format_stamp: "Formatted at 2026-09-24 07:17:56 on dist-test-slave-7q7m"
server_key: "6da9553bc7e274d6fc34b1cf2ec417f7"
server_key_iv: "3ead3adcc8f43c67ea144687b3b76874"
server_key_version: "encryptionkey@0"
I20260924 07:17:56.984670 31349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:56.997975 31349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:56.999902 31349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:57.000087 31349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:57.003595 31349 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36187
I20260924 07:17:57.003628 31416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36187 every 8 connection(s)
I20260924 07:17:57.003921 31349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:57.004693 31417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:57.006415 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 31349
I20260924 07:17:57.006529 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:17:57.006713 10763 external_mini_cluster.cc:1428] Setting key 47837f11edc85efcd61e9be504ee3ddd
I20260924 07:17:57.006980 31417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004: Bootstrap starting.
I20260924 07:17:57.007572 31417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:57.007817 31417 log.cc:824] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:57.008414 31417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004: No bootstrap required, opened a new log
I20260924 07:17:57.010088 31417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ddff71c974d74a26ec2375657e004" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36187 } }
I20260924 07:17:57.010212 31417 raft_consensus.cc:374] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:57.010232 31417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 330ddff71c974d74a26ec2375657e004, State: Initialized, Role: FOLLOWER
I20260924 07:17:57.010337 31417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ddff71c974d74a26ec2375657e004" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36187 } }
I20260924 07:17:57.010411 31417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:57.010452 31417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:57.010496 31417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:57.011060 31417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ddff71c974d74a26ec2375657e004" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36187 } }
I20260924 07:17:57.011144 31417 leader_election.cc:302] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 330ddff71c974d74a26ec2375657e004; no voters:
I20260924 07:17:57.011318 31417 leader_election.cc:288] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:17:57.011350 31422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:57.011495 31422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [term 1 LEADER]: Becoming Leader. State: Replica: 330ddff71c974d74a26ec2375657e004, State: Running, Role: LEADER
I20260924 07:17:57.011569 31417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:57.011584 31422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ddff71c974d74a26ec2375657e004" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36187 } }
I20260924 07:17:57.011656 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:57.007158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51663 (local address 127.0.0.1:36187)
0924 07:17:57.007329 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:57.007339 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:57.007365 (+ 26us) server_negotiation.cc:407] Connection header received
0924 07:17:57.007814 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:57.007820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:57.007872 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:57.008016 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:57.008235 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:57.009135 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:57.009874 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:57.010283 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:57.010372 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:57.010944 (+ 572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:57.010947 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:57.010948 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:57.011096 (+ 148us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:57.011129 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:57.011143 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:57.011203 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:57.011262 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 07:17:57.011409 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260924 07:17:57.012522 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:17:57.012720 31423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "330ddff71c974d74a26ec2375657e004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ddff71c974d74a26ec2375657e004" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36187 } } }
I20260924 07:17:57.012796 31423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:57.013082 31424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 330ddff71c974d74a26ec2375657e004. Latest consensus state: current_term: 1 leader_uuid: "330ddff71c974d74a26ec2375657e004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "330ddff71c974d74a26ec2375657e004" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36187 } } }
I20260924 07:17:57.013128 31424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004 [sys.catalog]: This master's current role is: LEADER
W20260924 07:17:57.013303 31437 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:17:57.013360 31437 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:17:57.016350 31420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:57.013934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52857 (local address 127.0.0.1:36187)
0924 07:17:57.013965 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:57.013966 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:57.014076 (+ 110us) server_negotiation.cc:407] Connection header received
0924 07:17:57.014109 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:57.014110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:57.014119 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:57.014160 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:57.014267 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:57.014707 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:57.015357 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:57.015564 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:57.015615 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:57.016168 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:57.016170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:57.016170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:57.016174 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:57.016185 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:57.016189 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:57.016230 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:57.016271 (+ 41us) server_negotiation.cc:299] Negotiation successful
0924 07:17:57.016289 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
I20260924 07:17:57.020021 31438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:57.020181 31438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:57.021391 31438 catalog_manager.cc:1347] Generated new cluster ID: 8f9186eb86e34b01aab8580f7168e5f4
I20260924 07:17:57.021442 31438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:17:57.036422 31438 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:17:57.037129 31438 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:57.045153 31438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 330ddff71c974d74a26ec2375657e004: Generated new TSK 0
I20260924 07:17:57.045338 31438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:17:57.047359 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 31349
2026-09-24T07:17:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1197 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:17:57 dist-test-slave-7q7m krb5kdc[31448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:17:57 dist-test-slave-7q7m krb5kdc[31448](info): set up 2 sockets
Sep 24 07:17:57 dist-test-slave-7q7m krb5kdc[31448](info): commencing operation
krb5kdc: starting...
W20260924 07:17:57.983644 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:17:58 dist-test-slave-7q7m krb5kdc[31448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:17:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:17:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 07:17:58.088286 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:58.164732 31461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:58.164927 31461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:58.164948 31461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:58.166072 31461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:58.166112 31461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:58.166126 31461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:58.166136 31461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:58.166157 31461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:58.167608 31461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:58.167820 31461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:58.168048 31461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:58.172466 31467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:58.172478 31466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:58.172478 31469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:58.172691 31461 server_base.cc:1034] running on GCE node
I20260924 07:17:58.172900 31461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:58.173178 31461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:58.174314 31461 hybrid_clock.cc:648] HybridClock initialized: now 1790234278174300 us; error 30 us; skew 500 ppm
Sep 24 07:17:58 dist-test-slave-7q7m krb5kdc[31448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234278, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:58.176553 31461 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 07:17:58.176928 31461 webserver.cc:460] Webserver started at http://127.0.0.1:40865/ using document root <none> and password file <none>
I20260924 07:17:58.177135 31461 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:58.177176 31461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:58.177289 31461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:58.178267 31461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "1c877334732843a9b7360a433edfa655"
format_stamp: "Formatted at 2026-09-24 07:17:58 on dist-test-slave-7q7m"
server_key: "36b908dec2384db31a7a27d3e9c5200f"
server_key_iv: "cede94c2367eba14cb3c956992ba0a72"
server_key_version: "encryptionkey@0"
I20260924 07:17:58.178581 31461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "1c877334732843a9b7360a433edfa655"
format_stamp: "Formatted at 2026-09-24 07:17:58 on dist-test-slave-7q7m"
server_key: "36b908dec2384db31a7a27d3e9c5200f"
server_key_iv: "cede94c2367eba14cb3c956992ba0a72"
server_key_version: "encryptionkey@0"
I20260924 07:17:58.179725 31461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:17:58.180343 31476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:58.180467 31461 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:58.180544 31461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "1c877334732843a9b7360a433edfa655"
format_stamp: "Formatted at 2026-09-24 07:17:58 on dist-test-slave-7q7m"
server_key: "36b908dec2384db31a7a27d3e9c5200f"
server_key_iv: "cede94c2367eba14cb3c956992ba0a72"
server_key_version: "encryptionkey@0"
I20260924 07:17:58.180605 31461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:58.183677 31461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:58.185274 31461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:58.185388 31461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:58.188903 31461 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38477
I20260924 07:17:58.188956 31528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38477 every 8 connection(s)
I20260924 07:17:58.189225 31461 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:58.190129 31529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:58.191958 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 31461
I20260924 07:17:58.192080 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:17:58.192240 10763 external_mini_cluster.cc:1428] Setting key 1c9322f4e812679930500df9c3ef0a25
I20260924 07:17:58.192235 31529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655: Bootstrap starting.
I20260924 07:17:58.192862 31529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:58.193152 31529 log.cc:824] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:58.193827 31529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655: No bootstrap required, opened a new log
I20260924 07:17:58.194635 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:58.192690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50777 (local address 127.0.0.1:38477)
0924 07:17:58.192891 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:58.192900 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:58.192933 (+ 33us) server_negotiation.cc:407] Connection header received
0924 07:17:58.193477 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:58.193484 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:58.193514 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:58.193654 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:58.194234 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:58.194237 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:58.194238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:58.194242 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:58.194288 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:58.194301 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:58.194391 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:58.194418 (+ 27us) server_negotiation.cc:299] Negotiation successful
0924 07:17:58.194454 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260924 07:17:58.195530 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:17:58.195935 31529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c877334732843a9b7360a433edfa655" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38477 } }
I20260924 07:17:58.196105 31529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:58.196130 31529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c877334732843a9b7360a433edfa655, State: Initialized, Role: FOLLOWER
I20260924 07:17:58.196239 31529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c877334732843a9b7360a433edfa655" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38477 } }
I20260924 07:17:58.196317 31529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:58.196341 31529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:58.196383 31529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:58.196883 31529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c877334732843a9b7360a433edfa655" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38477 } }
I20260924 07:17:58.196980 31529 leader_election.cc:302] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c877334732843a9b7360a433edfa655; no voters:
I20260924 07:17:58.197131 31529 leader_election.cc:288] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:17:58.197162 31539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:58.197324 31539 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [term 1 LEADER]: Becoming Leader. State: Replica: 1c877334732843a9b7360a433edfa655, State: Running, Role: LEADER
I20260924 07:17:58.197359 31529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:58.197443 31539 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c877334732843a9b7360a433edfa655" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38477 } }
I20260924 07:17:58.197788 31539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c877334732843a9b7360a433edfa655. Latest consensus state: current_term: 1 leader_uuid: "1c877334732843a9b7360a433edfa655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c877334732843a9b7360a433edfa655" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38477 } } }
I20260924 07:17:58.198017 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:58.196826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49117 (local address 127.0.0.1:38477)
0924 07:17:58.196872 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:58.196873 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:58.196978 (+ 105us) server_negotiation.cc:407] Connection header received
0924 07:17:58.197029 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:58.197030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:58.197034 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:58.197108 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:58.197652 (+ 544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:58.197653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:58.197653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:17:58.197657 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:58.197671 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:17:58.197675 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:58.197730 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:58.197915 (+ 185us) server_negotiation.cc:299] Negotiation successful
0924 07:17:58.197928 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 07:17:58.198086 31539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:58.197932 31541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c877334732843a9b7360a433edfa655" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c877334732843a9b7360a433edfa655" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38477 } } }
I20260924 07:17:58.198230 31541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655 [sys.catalog]: This master's current role is: LEADER
W20260924 07:17:58.199051 31555 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:17:58.199093 31555 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:17:58.199150 31548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:58.199323 31548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:58.200765 31548 catalog_manager.cc:1347] Generated new cluster ID: 5782807fa9db486cb95db0a869226327
I20260924 07:17:58.200814 31548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:17:58.208252 31548 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:17:58.208388 31548 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:58.216064 31548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c877334732843a9b7360a433edfa655: Generated new TSK 0
I20260924 07:17:58.216221 31548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:17:58.222918 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 31461
2026-09-24T07:17:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1175 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:17:58 dist-test-slave-7q7m krb5kdc[31560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:17:58 dist-test-slave-7q7m krb5kdc[31560](info): set up 2 sockets
Sep 24 07:17:58 dist-test-slave-7q7m krb5kdc[31560](info): commencing operation
krb5kdc: starting...
W20260924 07:17:59.163767 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.924s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:17:59 dist-test-slave-7q7m krb5kdc[31560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:17:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:17:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 07:17:59.267697 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45851
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:17:59.343699 31573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:17:59.343894 31573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:17:59.343923 31573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:17:59.345049 31573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:17:59.345089 31573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:17:59.345108 31573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:17:59.345124 31573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:17:59.345137 31573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:17:59.346545 31573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:17:59.346786 31573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:17:59.346975 31573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:17:59.351342 31578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:59.351402 31579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:17:59.351342 31581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:17:59.351480 31573 server_base.cc:1034] running on GCE node
I20260924 07:17:59.351799 31573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:17:59.352082 31573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:17:59.353217 31573 hybrid_clock.cc:648] HybridClock initialized: now 1790234279353203 us; error 31 us; skew 500 ppm
Sep 24 07:17:59 dist-test-slave-7q7m krb5kdc[31560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:17:59.355456 31573 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 07:17:59.355859 31573 webserver.cc:460] Webserver started at http://127.0.0.1:34489/ using document root <none> and password file <none>
I20260924 07:17:59.356088 31573 fs_manager.cc:359] Metadata directory not provided
I20260924 07:17:59.356139 31573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:17:59.356257 31573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:17:59.357236 31573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "0249ffa54ce54474a0056ee123438a29"
format_stamp: "Formatted at 2026-09-24 07:17:59 on dist-test-slave-7q7m"
server_key: "f6f0903d598c2fe1e79812f5c4e7c66f"
server_key_iv: "96992d531774365df895a4d7d308252e"
server_key_version: "encryptionkey@0"
I20260924 07:17:59.357554 31573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "0249ffa54ce54474a0056ee123438a29"
format_stamp: "Formatted at 2026-09-24 07:17:59 on dist-test-slave-7q7m"
server_key: "f6f0903d598c2fe1e79812f5c4e7c66f"
server_key_iv: "96992d531774365df895a4d7d308252e"
server_key_version: "encryptionkey@0"
I20260924 07:17:59.358757 31573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 07:17:59.359401 31588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:17:59.359573 31573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:17:59.359658 31573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "0249ffa54ce54474a0056ee123438a29"
format_stamp: "Formatted at 2026-09-24 07:17:59 on dist-test-slave-7q7m"
server_key: "f6f0903d598c2fe1e79812f5c4e7c66f"
server_key_iv: "96992d531774365df895a4d7d308252e"
server_key_version: "encryptionkey@0"
I20260924 07:17:59.359717 31573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:17:59.367466 31573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:17:59.369369 31573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:17:59.369510 31573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:17:59.373248 31573 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45851
I20260924 07:17:59.373288 31640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45851 every 8 connection(s)
I20260924 07:17:59.373610 31573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:17:59.374395 31641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:17:59.376605 31641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29: Bootstrap starting.
I20260924 07:17:59.377105 31641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29: Neither blocks nor log segments found. Creating new log.
I20260924 07:17:59.377321 31641 log.cc:824] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29: Log is configured to *not* fsync() on all Append() calls
I20260924 07:17:59.377944 31641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29: No bootstrap required, opened a new log
I20260924 07:17:59.379451 31641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0249ffa54ce54474a0056ee123438a29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45851 } }
I20260924 07:17:59.379607 31641 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:17:59.379630 31641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0249ffa54ce54474a0056ee123438a29, State: Initialized, Role: FOLLOWER
I20260924 07:17:59.379746 31641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0249ffa54ce54474a0056ee123438a29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45851 } }
I20260924 07:17:59.379819 31641 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:17:59.379845 31641 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:17:59.379877 31641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:17:59.380378 31641 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0249ffa54ce54474a0056ee123438a29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45851 } }
I20260924 07:17:59.380462 31641 leader_election.cc:302] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0249ffa54ce54474a0056ee123438a29; no voters:
I20260924 07:17:59.380589 31641 leader_election.cc:288] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:17:59.380635 31644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:17:59.380803 31641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:17:59.380806 31644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [term 1 LEADER]: Becoming Leader. State: Replica: 0249ffa54ce54474a0056ee123438a29, State: Running, Role: LEADER
I20260924 07:17:59.380962 31644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0249ffa54ce54474a0056ee123438a29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45851 } }
I20260924 07:17:59.381283 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 31573
I20260924 07:17:59.381319 31646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0249ffa54ce54474a0056ee123438a29. Latest consensus state: current_term: 1 leader_uuid: "0249ffa54ce54474a0056ee123438a29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0249ffa54ce54474a0056ee123438a29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45851 } } }
I20260924 07:17:59.381385 31646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:59.381410 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:17:59.381512 31645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0249ffa54ce54474a0056ee123438a29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0249ffa54ce54474a0056ee123438a29" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45851 } } }
I20260924 07:17:59.381556 31645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29 [sys.catalog]: This master's current role is: LEADER
I20260924 07:17:59.381587 10763 external_mini_cluster.cc:1428] Setting key dcdaba1773a605cbcdb238dfeecdec45
I20260924 07:17:59.381906 31648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:17:59.382465 31648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:17:59.384375 31648 catalog_manager.cc:1347] Generated new cluster ID: 49b8c1b85b244efba9d0c0a5d85deee4
I20260924 07:17:59.384425 31648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:17:59 dist-test-slave-7q7m krb5kdc[31560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 07:17:59.393095 31662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:59.382234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.122:41355 (local address 127.0.0.1:45851)
0924 07:17:59.382418 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:59.382427 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:59.382457 (+ 30us) server_negotiation.cc:407] Connection header received
0924 07:17:59.382570 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:59.382577 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:59.382767 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:59.383573 (+ 806us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:59.383652 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:59.384883 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:59.385980 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:59.386576 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:59.386667 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:59.389388 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:59.389392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:59.389392 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:59.389404 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:59.391295 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:59.391705 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:59.391708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:59.391709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:59.391827 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:59.392057 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:59.392058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:59.392058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:59.392376 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:59.392527 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:59.392632 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 07:17:59.392848 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260924 07:17:59.394234 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:17:59.400184 31648 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:17:59.400655 31648 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:17:59.403188 31662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:17:59.395618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.122:49357 (local address 127.0.0.1:45851)
0924 07:17:59.395650 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 07:17:59.395884 (+ 234us) server_negotiation.cc:399] Waiting for connection header
0924 07:17:59.396026 (+ 142us) server_negotiation.cc:407] Connection header received
0924 07:17:59.396127 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:17:59.396128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:17:59.396138 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:17:59.396164 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:17:59.396798 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:59.397585 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:59.398667 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:17:59.398980 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:17:59.399018 (+ 38us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:17:59.400432 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:17:59.400433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:17:59.400433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:17:59.400439 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:17:59.401910 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:59.402319 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:59.402320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:59.402320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:59.402372 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:17:59.402607 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:17:59.402608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:17:59.402608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:17:59.402724 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:17:59.402810 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 07:17:59.403030 (+ 220us) server_negotiation.cc:299] Negotiation successful
0924 07:17:59.403103 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:17:59.404624 31648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0249ffa54ce54474a0056ee123438a29: Generated new TSK 0
I20260924 07:17:59.404786 31648 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:17:59.406659 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 31573
W20260924 07:17:59.410125 31558 connection.cc:535] client connection to 127.0.0.1:45851 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45851): Connection reset by peer (error 104)
2026-09-24T07:17:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1183 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:17:59 dist-test-slave-7q7m krb5kdc[31672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:17:59 dist-test-slave-7q7m krb5kdc[31672](info): set up 2 sockets
Sep 24 07:17:59 dist-test-slave-7q7m krb5kdc[31672](info): commencing operation
krb5kdc: starting...
W20260924 07:18:00.351603 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:18:00 dist-test-slave-7q7m krb5kdc[31672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:18:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:18:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 07:18:00.456497 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34309
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:18:00.532829 31685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:00.533025 31685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:00.533046 31685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:00.534118 31685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:00.534153 31685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:00.534166 31685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:00.534178 31685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:00.534188 31685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:00.535573 31685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:00.535761 31685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:00.535928 31685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:00.540444 31693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:00.540443 31690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:00.540455 31691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:00.540661 31685 server_base.cc:1034] running on GCE node
I20260924 07:18:00.540858 31685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:00.541121 31685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:00.542266 31685 hybrid_clock.cc:648] HybridClock initialized: now 1790234280542246 us; error 34 us; skew 500 ppm
Sep 24 07:18:00 dist-test-slave-7q7m krb5kdc[31672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234280, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:00.544516 31685 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 07:18:00.544936 31685 webserver.cc:460] Webserver started at http://127.0.0.1:34505/ using document root <none> and password file <none>
I20260924 07:18:00.545131 31685 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:00.545177 31685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:00.545295 31685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:00.546277 31685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "93ff6b0ad59f411c887ae7c8713bc3c8"
format_stamp: "Formatted at 2026-09-24 07:18:00 on dist-test-slave-7q7m"
server_key: "2e16f2d2779027baed527962c3c1072b"
server_key_iv: "1cc781f3af117171e63d813c36f0e3f5"
server_key_version: "encryptionkey@0"
I20260924 07:18:00.546588 31685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "93ff6b0ad59f411c887ae7c8713bc3c8"
format_stamp: "Formatted at 2026-09-24 07:18:00 on dist-test-slave-7q7m"
server_key: "2e16f2d2779027baed527962c3c1072b"
server_key_iv: "1cc781f3af117171e63d813c36f0e3f5"
server_key_version: "encryptionkey@0"
I20260924 07:18:00.547737 31685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:18:00.548400 31700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:00.548538 31685 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:00.548614 31685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "93ff6b0ad59f411c887ae7c8713bc3c8"
format_stamp: "Formatted at 2026-09-24 07:18:00 on dist-test-slave-7q7m"
server_key: "2e16f2d2779027baed527962c3c1072b"
server_key_iv: "1cc781f3af117171e63d813c36f0e3f5"
server_key_version: "encryptionkey@0"
I20260924 07:18:00.548683 31685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:00.565621 31685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:00.567178 31685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:00.567302 31685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:00.570966 31685 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34309
I20260924 07:18:00.571005 31752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34309 every 8 connection(s)
I20260924 07:18:00.571300 31685 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:18:00.572132 31753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:00.574199 31753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8: Bootstrap starting.
I20260924 07:18:00.574714 31753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:00.574949 31753 log.cc:824] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:00.575534 31753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8: No bootstrap required, opened a new log
I20260924 07:18:00.577090 31753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93ff6b0ad59f411c887ae7c8713bc3c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34309 } }
I20260924 07:18:00.577222 31753 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:18:00.577244 31753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93ff6b0ad59f411c887ae7c8713bc3c8, State: Initialized, Role: FOLLOWER
I20260924 07:18:00.577376 31753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93ff6b0ad59f411c887ae7c8713bc3c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34309 } }
I20260924 07:18:00.577447 31753 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:18:00.577479 31753 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:18:00.577524 31753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:00.577998 31753 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93ff6b0ad59f411c887ae7c8713bc3c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34309 } }
I20260924 07:18:00.578083 31753 leader_election.cc:302] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93ff6b0ad59f411c887ae7c8713bc3c8; no voters:
I20260924 07:18:00.578227 31753 leader_election.cc:288] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:18:00.578264 31756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:18:00.578405 31756 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [term 1 LEADER]: Becoming Leader. State: Replica: 93ff6b0ad59f411c887ae7c8713bc3c8, State: Running, Role: LEADER
I20260924 07:18:00.578450 31753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:00.578491 31756 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93ff6b0ad59f411c887ae7c8713bc3c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34309 } }
I20260924 07:18:00.578758 31756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93ff6b0ad59f411c887ae7c8713bc3c8. Latest consensus state: current_term: 1 leader_uuid: "93ff6b0ad59f411c887ae7c8713bc3c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93ff6b0ad59f411c887ae7c8713bc3c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34309 } } }
I20260924 07:18:00.578786 31757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93ff6b0ad59f411c887ae7c8713bc3c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93ff6b0ad59f411c887ae7c8713bc3c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34309 } } }
I20260924 07:18:00.578837 31756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:00.578847 31757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8 [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:00.579159 31762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:18:00.579614 31762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:18:00.580262 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 31685
I20260924 07:18:00.580381 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:18:00.580554 10763 external_mini_cluster.cc:1428] Setting key 043cd8f85dba0d90c7785348e9eb2d01
I20260924 07:18:00.581140 31762 catalog_manager.cc:1347] Generated new cluster ID: d4d2839403ec46739d9311e113fc6df9
I20260924 07:18:00.581193 31762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:18:00.588097 31776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:00.580962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.122:40271 (local address 127.0.0.1:34309)
0924 07:18:00.584085 (+ 3123us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:00.584093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:00.584240 (+ 147us) server_negotiation.cc:407] Connection header received
0924 07:18:00.584370 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:00.584376 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:00.584419 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:00.584532 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:00.584720 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:00.585621 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:00.586358 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:00.586781 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:00.586870 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:00.587511 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:00.587515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:00.587515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:00.587668 (+ 153us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:00.587697 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:00.587707 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:00.587772 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:00.587861 (+ 89us) server_negotiation.cc:299] Negotiation successful
0924 07:18:00.587888 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3077,"thread_start_us":56,"threads_started":1}
I20260924 07:18:00.588863 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:18:00.594785 31776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:00.592268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.122:37725 (local address 127.0.0.1:34309)
0924 07:18:00.592302 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:00.592303 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:00.592436 (+ 133us) server_negotiation.cc:407] Connection header received
0924 07:18:00.592485 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:00.592486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:00.592496 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:00.592538 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:00.592652 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:00.593105 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:00.593752 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:00.593952 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:00.594001 (+ 49us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:00.594579 (+ 578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:00.594580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:00.594580 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:00.594584 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:00.594596 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:00.594600 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:00.594643 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:00.594708 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 07:18:00.594719 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260924 07:18:00.599390 31762 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:18:00.600073 31762 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:18:00.604038 31762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93ff6b0ad59f411c887ae7c8713bc3c8: Generated new TSK 0
I20260924 07:18:00.604192 31762 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:18:00.610735 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 31685
W20260924 07:18:00.614348 31670 connection.cc:535] client connection to 127.0.0.1:34309 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34309): Connection reset by peer (error 104)
2026-09-24T07:18:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1204 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:18:00 dist-test-slave-7q7m krb5kdc[31784](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:18:00 dist-test-slave-7q7m krb5kdc[31784](info): set up 2 sockets
Sep 24 07:18:00 dist-test-slave-7q7m krb5kdc[31784](info): commencing operation
krb5kdc: starting...
W20260924 07:18:01.555629 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.928s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:18:01 dist-test-slave-7q7m krb5kdc[31784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:18:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:18:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 07:18:01.660681 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38311
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:18:01.737407 31797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:01.737597 31797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:01.737627 31797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:01.738719 31797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:01.738758 31797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:01.738778 31797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:01.738794 31797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:01.738809 31797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:01.740236 31797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:01.740463 31797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:01.740651 31797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 07:18:01.745023 31797 server_base.cc:1034] running on GCE node
W20260924 07:18:01.744989 31803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:01.744978 31802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:01.744985 31805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:01.745378 31797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:01.745662 31797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:01.746798 31797 hybrid_clock.cc:648] HybridClock initialized: now 1790234281746784 us; error 26 us; skew 500 ppm
Sep 24 07:18:01 dist-test-slave-7q7m krb5kdc[31784](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234281, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:01.753890 31797 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 07:18:01.754302 31797 webserver.cc:460] Webserver started at http://127.0.0.1:33059/ using document root <none> and password file <none>
I20260924 07:18:01.754501 31797 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:01.754549 31797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:01.754680 31797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:01.755694 31797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "03b6075cbc6a43f882d77b3225a7b08f"
format_stamp: "Formatted at 2026-09-24 07:18:01 on dist-test-slave-7q7m"
server_key: "d8dbf14d21d7ad8db518d0faacdc2b19"
server_key_iv: "992bde464fce591cfad261b508ee00df"
server_key_version: "encryptionkey@0"
I20260924 07:18:01.756068 31797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "03b6075cbc6a43f882d77b3225a7b08f"
format_stamp: "Formatted at 2026-09-24 07:18:01 on dist-test-slave-7q7m"
server_key: "d8dbf14d21d7ad8db518d0faacdc2b19"
server_key_iv: "992bde464fce591cfad261b508ee00df"
server_key_version: "encryptionkey@0"
I20260924 07:18:01.757296 31797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 07:18:01.757956 31812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:01.758103 31797 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:18:01.758185 31797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "03b6075cbc6a43f882d77b3225a7b08f"
format_stamp: "Formatted at 2026-09-24 07:18:01 on dist-test-slave-7q7m"
server_key: "d8dbf14d21d7ad8db518d0faacdc2b19"
server_key_iv: "992bde464fce591cfad261b508ee00df"
server_key_version: "encryptionkey@0"
I20260924 07:18:01.758250 31797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:01.761435 31797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:01.763084 31797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:01.763204 31797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:01.766700 31797 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38311
I20260924 07:18:01.766727 31864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38311 every 8 connection(s)
I20260924 07:18:01.766988 31797 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:18:01.767907 31865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:01.770202 31865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f: Bootstrap starting.
I20260924 07:18:01.770720 31865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:01.770956 31865 log.cc:824] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:01.771564 31865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f: No bootstrap required, opened a new log
I20260924 07:18:01.773088 31865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03b6075cbc6a43f882d77b3225a7b08f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } }
I20260924 07:18:01.773211 31865 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:18:01.773240 31865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03b6075cbc6a43f882d77b3225a7b08f, State: Initialized, Role: FOLLOWER
I20260924 07:18:01.773370 31865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03b6075cbc6a43f882d77b3225a7b08f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } }
I20260924 07:18:01.773442 31865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:18:01.773476 31865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:18:01.773521 31865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:01.773988 31865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03b6075cbc6a43f882d77b3225a7b08f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } }
I20260924 07:18:01.774075 31865 leader_election.cc:302] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03b6075cbc6a43f882d77b3225a7b08f; no voters:
I20260924 07:18:01.774217 31865 leader_election.cc:288] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:18:01.774242 31868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:18:01.774384 31868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [term 1 LEADER]: Becoming Leader. State: Replica: 03b6075cbc6a43f882d77b3225a7b08f, State: Running, Role: LEADER
I20260924 07:18:01.774447 31865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:01.774487 31868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03b6075cbc6a43f882d77b3225a7b08f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } }
I20260924 07:18:01.774585 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 31797
I20260924 07:18:01.774727 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:18:01.774904 10763 external_mini_cluster.cc:1428] Setting key f2f1db670bfd87a79f32fad086f60133
I20260924 07:18:01.775511 31870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03b6075cbc6a43f882d77b3225a7b08f. Latest consensus state: current_term: 1 leader_uuid: "03b6075cbc6a43f882d77b3225a7b08f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03b6075cbc6a43f882d77b3225a7b08f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } } }
I20260924 07:18:01.775596 31870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:01.775765 31869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03b6075cbc6a43f882d77b3225a7b08f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03b6075cbc6a43f882d77b3225a7b08f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } } }
I20260924 07:18:01.776075 31869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f [sys.catalog]: This master's current role is: LEADER
W20260924 07:18:01.776417 31886 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:18:01.776467 31886 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:18:01.776515 31885 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:18:01.776656 31885 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:18:01.777350 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:01.775521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.122:51145 (local address 127.0.0.1:38311)
0924 07:18:01.775960 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:01.775969 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:01.776016 (+ 47us) server_negotiation.cc:407] Connection header received
0924 07:18:01.776135 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:01.776142 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:01.776171 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:01.776292 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:01.776929 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:01.776933 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:01.776933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:01.776937 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:01.776977 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:01.776988 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:01.777049 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:01.777107 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 07:18:01.777225 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":73,"threads_started":1}
I20260924 07:18:01.778124 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:18:01.780242 31883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:01.779341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.122:47387 (local address 127.0.0.1:38311)
0924 07:18:01.779390 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:01.779391 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:01.779471 (+ 80us) server_negotiation.cc:407] Connection header received
0924 07:18:01.779546 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:01.779547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:01.779551 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:01.779591 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:01.780047 (+ 456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:01.780047 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:01.780048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:01.780051 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:01.780064 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:01.780067 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:01.780106 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:01.780165 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 07:18:01.780175 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 07:18:01.784390 31885 catalog_manager.cc:1347] Generated new cluster ID: 9d21073f0e634f558e98320f6ec32727
I20260924 07:18:01.784432 31885 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:18:01.792678 31885 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:18:01.792809 31885 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:18:01.803479 31885 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03b6075cbc6a43f882d77b3225a7b08f: Generated new TSK 0
I20260924 07:18:01.803653 31885 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:18:01.807646 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 31797
2026-09-24T07:18:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1197 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:18:01 dist-test-slave-7q7m krb5kdc[31896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:18:01 dist-test-slave-7q7m krb5kdc[31896](info): set up 2 sockets
Sep 24 07:18:01 dist-test-slave-7q7m krb5kdc[31896](info): commencing operation
krb5kdc: starting...
W20260924 07:18:02.743687 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.919s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:18:02 dist-test-slave-7q7m krb5kdc[31896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:18:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:18:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 07:18:02.852415 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39671
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:18:02.928246 31909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:02.928431 31909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:02.928452 31909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:02.929574 31909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:02.929612 31909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:02.929625 31909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:02.929637 31909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:02.929661 31909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:02.931066 31909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39671
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:02.931274 31909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:02.931463 31909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:02.935842 31914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:02.935931 31909 server_base.cc:1034] running on GCE node
W20260924 07:18:02.935863 31915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:02.936069 31917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:02.936323 31909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:02.936617 31909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:02.937775 31909 hybrid_clock.cc:648] HybridClock initialized: now 1790234282937753 us; error 33 us; skew 500 ppm
Sep 24 07:18:02 dist-test-slave-7q7m krb5kdc[31896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234282, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:02.939957 31909 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 07:18:02.940369 31909 webserver.cc:460] Webserver started at http://127.0.0.1:38837/ using document root <none> and password file <none>
I20260924 07:18:02.940549 31909 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:02.940603 31909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:02.940724 31909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:02.941704 31909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "e0510f8c82df49fdbc5bcd4556e416b1"
format_stamp: "Formatted at 2026-09-24 07:18:02 on dist-test-slave-7q7m"
server_key: "d859af0494fe5feb25b9ef093458c81c"
server_key_iv: "9bfb132ac4466c535a1e301fc07c8e6d"
server_key_version: "encryptionkey@0"
I20260924 07:18:02.942010 31909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "e0510f8c82df49fdbc5bcd4556e416b1"
format_stamp: "Formatted at 2026-09-24 07:18:02 on dist-test-slave-7q7m"
server_key: "d859af0494fe5feb25b9ef093458c81c"
server_key_iv: "9bfb132ac4466c535a1e301fc07c8e6d"
server_key_version: "encryptionkey@0"
I20260924 07:18:02.943161 31909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:18:02.943735 31924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:02.943887 31909 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:18:02.943964 31909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "e0510f8c82df49fdbc5bcd4556e416b1"
format_stamp: "Formatted at 2026-09-24 07:18:02 on dist-test-slave-7q7m"
server_key: "d859af0494fe5feb25b9ef093458c81c"
server_key_iv: "9bfb132ac4466c535a1e301fc07c8e6d"
server_key_version: "encryptionkey@0"
I20260924 07:18:02.944047 31909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:02.958034 31909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:02.959744 31909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:02.959869 31909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:02.963447 31909 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39671
I20260924 07:18:02.963486 31976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39671 every 8 connection(s)
I20260924 07:18:02.963792 31909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:18:02.964651 31977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:02.965939 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 31909
I20260924 07:18:02.966037 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:18:02.966207 10763 external_mini_cluster.cc:1428] Setting key f273852ebed475c10f93c5231e72e236
I20260924 07:18:02.967183 31977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1: Bootstrap starting.
I20260924 07:18:02.967857 31977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:02.968150 31977 log.cc:824] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:02.968758 31977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1: No bootstrap required, opened a new log
I20260924 07:18:02.970422 31977 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0510f8c82df49fdbc5bcd4556e416b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39671 } }
I20260924 07:18:02.970561 31977 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:18:02.970595 31977 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0510f8c82df49fdbc5bcd4556e416b1, State: Initialized, Role: FOLLOWER
I20260924 07:18:02.970713 31977 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0510f8c82df49fdbc5bcd4556e416b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39671 } }
I20260924 07:18:02.970791 31977 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:18:02.970826 31977 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:18:02.970878 31977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:02.971371 31977 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0510f8c82df49fdbc5bcd4556e416b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39671 } }
I20260924 07:18:02.971450 31977 leader_election.cc:302] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0510f8c82df49fdbc5bcd4556e416b1; no voters:
I20260924 07:18:02.971597 31977 leader_election.cc:288] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:18:02.971629 31982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:18:02.971776 31982 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [term 1 LEADER]: Becoming Leader. State: Replica: e0510f8c82df49fdbc5bcd4556e416b1, State: Running, Role: LEADER
I20260924 07:18:02.971844 31977 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:02.971899 31982 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0510f8c82df49fdbc5bcd4556e416b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39671 } }
Sep 24 07:18:02 dist-test-slave-7q7m krb5kdc[31896](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 07:18:02.972280 31982 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0510f8c82df49fdbc5bcd4556e416b1. Latest consensus state: current_term: 1 leader_uuid: "e0510f8c82df49fdbc5bcd4556e416b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0510f8c82df49fdbc5bcd4556e416b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39671 } } }
I20260924 07:18:02.972357 31982 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:02.972494 31984 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0510f8c82df49fdbc5bcd4556e416b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0510f8c82df49fdbc5bcd4556e416b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39671 } } }
I20260924 07:18:02.972602 31984 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1 [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:02.973312 31989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:18:02.973743 31989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:18:02.975262 31989 catalog_manager.cc:1347] Generated new cluster ID: b30490404f0d43cd867f6d7dec366e42
I20260924 07:18:02.975306 31989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:18:02.976174 31980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:02.966716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.122:49863 (local address 127.0.0.1:39671)
0924 07:18:02.966897 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:02.966907 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:02.966938 (+ 31us) server_negotiation.cc:407] Connection header received
0924 07:18:02.967441 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:02.967451 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:02.967523 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:02.967673 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:02.968270 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:02.969181 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:02.969882 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:02.970264 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:02.970356 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:02.972893 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:02.972897 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:02.972898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:02.972910 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:02.974528 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:02.974958 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:02.974961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:02.974961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:02.975040 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:02.975273 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:02.975274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:02.975275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:02.975568 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:02.975694 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:02.975864 (+ 170us) server_negotiation.cc:299] Negotiation successful
0924 07:18:02.975957 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":72,"threads_started":1}
I20260924 07:18:02.977036 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:18:02.986687 31989 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:18:02.986768 31980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:02.978390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.122:56771 (local address 127.0.0.1:39671)
0924 07:18:02.978427 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:02.978429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:02.980169 (+ 1740us) server_negotiation.cc:407] Connection header received
0924 07:18:02.980229 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:02.980230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:02.980240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:02.980278 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:02.980850 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:02.981546 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:02.982501 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:02.982793 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:02.982843 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:02.984173 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:02.984174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:02.984175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:02.984181 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:02.985601 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:02.985991 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:02.985992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:02.985992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:02.986046 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:02.986256 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:02.986257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:02.986257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:02.986370 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:02.986464 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:02.986594 (+ 130us) server_negotiation.cc:299] Negotiation successful
0924 07:18:02.986653 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:18:02.987316 31989 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:18:02.995373 31989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0510f8c82df49fdbc5bcd4556e416b1: Generated new TSK 0
I20260924 07:18:02.995519 31989 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:18:03.002733 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 31909
W20260924 07:18:03.006309 31894 connection.cc:535] client connection to 127.0.0.1:39671 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39671): Connection reset by peer (error 104)
2026-09-24T07:18:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1194 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:18:03 dist-test-slave-7q7m krb5kdc[32008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:18:03 dist-test-slave-7q7m krb5kdc[32008](info): set up 2 sockets
Sep 24 07:18:03 dist-test-slave-7q7m krb5kdc[32008](info): commencing operation
krb5kdc: starting...
W20260924 07:18:03.939641 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.920s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:18:04 dist-test-slave-7q7m krb5kdc[32008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:18:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:18:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 07:18:04.045881 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39109
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:18:04.124173 32021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:04.124360 32021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:04.124383 32021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:04.125461 32021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:04.125494 32021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:04.125507 32021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:04.125519 32021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:04.125528 32021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:04.126916 32021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39109
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:04.127101 32021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:04.127259 32021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:04.131774 32029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:04.131791 32027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:04.131774 32026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:04.131951 32021 server_base.cc:1034] running on GCE node
I20260924 07:18:04.132187 32021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:04.132453 32021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:04.133610 32021 hybrid_clock.cc:648] HybridClock initialized: now 1790234284133596 us; error 26 us; skew 500 ppm
Sep 24 07:18:04 dist-test-slave-7q7m krb5kdc[32008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234284, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:04.135859 32021 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 07:18:04.136304 32021 webserver.cc:460] Webserver started at http://127.0.0.1:41085/ using document root <none> and password file <none>
I20260924 07:18:04.136507 32021 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:04.136552 32021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:04.136691 32021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:04.137673 32021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "6b6e1a4fa0f149c0ac191dd3bb32e31e"
format_stamp: "Formatted at 2026-09-24 07:18:04 on dist-test-slave-7q7m"
server_key: "c28862428d954acd271cc291a6cc39f4"
server_key_iv: "2c3876f5cb12488e2def7468307c0aa3"
server_key_version: "encryptionkey@0"
I20260924 07:18:04.137990 32021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "6b6e1a4fa0f149c0ac191dd3bb32e31e"
format_stamp: "Formatted at 2026-09-24 07:18:04 on dist-test-slave-7q7m"
server_key: "c28862428d954acd271cc291a6cc39f4"
server_key_iv: "2c3876f5cb12488e2def7468307c0aa3"
server_key_version: "encryptionkey@0"
I20260924 07:18:04.139173 32021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:18:04.139794 32036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:04.139995 32021 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:18:04.140085 32021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "6b6e1a4fa0f149c0ac191dd3bb32e31e"
format_stamp: "Formatted at 2026-09-24 07:18:04 on dist-test-slave-7q7m"
server_key: "c28862428d954acd271cc291a6cc39f4"
server_key_iv: "2c3876f5cb12488e2def7468307c0aa3"
server_key_version: "encryptionkey@0"
I20260924 07:18:04.140151 32021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:04.150367 32021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:04.152087 32021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:04.152215 32021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:04.155756 32021 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39109
I20260924 07:18:04.155791 32088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39109 every 8 connection(s)
I20260924 07:18:04.156140 32021 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:18:04.156909 32089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:04.159106 32089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e: Bootstrap starting.
I20260924 07:18:04.159606 32089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:04.159874 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 32021
I20260924 07:18:04.159888 32089 log.cc:824] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:04.160007 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:18:04.160197 10763 external_mini_cluster.cc:1428] Setting key e8a24868a7bf60e70d36e8bb8ce613de
I20260924 07:18:04.160632 32089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e: No bootstrap required, opened a new log
I20260924 07:18:04.162951 32089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b6e1a4fa0f149c0ac191dd3bb32e31e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39109 } }
I20260924 07:18:04.163089 32089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:18:04.163122 32089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b6e1a4fa0f149c0ac191dd3bb32e31e, State: Initialized, Role: FOLLOWER
I20260924 07:18:04.163276 32089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b6e1a4fa0f149c0ac191dd3bb32e31e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39109 } }
I20260924 07:18:04.163358 32089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:18:04.163395 32089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:18:04.163437 32089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:04.164132 32089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b6e1a4fa0f149c0ac191dd3bb32e31e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39109 } }
I20260924 07:18:04.164237 32089 leader_election.cc:302] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b6e1a4fa0f149c0ac191dd3bb32e31e; no voters:
I20260924 07:18:04.164383 32089 leader_election.cc:288] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:18:04.164417 32094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:18:04.164558 32094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [term 1 LEADER]: Becoming Leader. State: Replica: 6b6e1a4fa0f149c0ac191dd3bb32e31e, State: Running, Role: LEADER
I20260924 07:18:04.164623 32089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:04.164721 32094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b6e1a4fa0f149c0ac191dd3bb32e31e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39109 } }
I20260924 07:18:04.165814 32095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b6e1a4fa0f149c0ac191dd3bb32e31e. Latest consensus state: current_term: 1 leader_uuid: "6b6e1a4fa0f149c0ac191dd3bb32e31e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b6e1a4fa0f149c0ac191dd3bb32e31e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39109 } } }
I20260924 07:18:04.165898 32095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:04.166129 32107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:18:04.166309 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:04.160672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.122:52767 (local address 127.0.0.1:39109)
0924 07:18:04.160833 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:04.160841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:04.160865 (+ 24us) server_negotiation.cc:407] Connection header received
0924 07:18:04.161767 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:04.161774 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:04.161823 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:04.161932 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:04.162154 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:04.163081 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:04.164162 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:04.164762 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:04.164908 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:04.165522 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:04.165525 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:04.165525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:04.165748 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:04.165777 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:04.165788 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:04.165862 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:04.166105 (+ 243us) server_negotiation.cc:299] Negotiation successful
0924 07:18:04.166132 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260924 07:18:04.166316 32096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b6e1a4fa0f149c0ac191dd3bb32e31e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b6e1a4fa0f149c0ac191dd3bb32e31e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39109 } } }
I20260924 07:18:04.166486 32096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:04.166774 32107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:18:04.167253 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:18:04.168030 32107 catalog_manager.cc:1347] Generated new cluster ID: 9045863e16bc4ae791df47a595521429
I20260924 07:18:04.168075 32107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:18:04.175594 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:04.172296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.122:58679 (local address 127.0.0.1:39109)
0924 07:18:04.172337 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:04.172339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:04.172438 (+ 99us) server_negotiation.cc:407] Connection header received
0924 07:18:04.172478 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:04.172479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:04.172488 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:04.172554 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:04.172669 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:04.173410 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:04.174360 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:04.174665 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:04.174731 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:04.175370 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:04.175372 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:04.175372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:04.175377 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:04.175391 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:04.175396 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:04.175456 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:04.175511 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 07:18:04.175520 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 07:18:04.177721 32107 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:18:04.178382 32107 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:18:04.189040 32107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b6e1a4fa0f149c0ac191dd3bb32e31e: Generated new TSK 0
I20260924 07:18:04.189240 32107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:18:04.199692 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 32021
W20260924 07:18:04.203434 32006 connection.cc:535] client connection to 127.0.0.1:39109 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39109): Connection reset by peer (error 104)
2026-09-24T07:18:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1197 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:18:04 dist-test-slave-7q7m krb5kdc[32120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:18:04 dist-test-slave-7q7m krb5kdc[32120](info): set up 2 sockets
Sep 24 07:18:04 dist-test-slave-7q7m krb5kdc[32120](info): commencing operation
krb5kdc: starting...
W20260924 07:18:05.143715 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.927s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:18:05 dist-test-slave-7q7m krb5kdc[32120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:18:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:18:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 07:18:05.248803 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37741
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:18:05.324911 32133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:05.325096 32133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:05.325116 32133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:05.326193 32133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:05.326225 32133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:05.326237 32133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:05.326248 32133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:05.326259 32133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:05.327616 32133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37741
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:05.327805 32133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:05.327968 32133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:05.332468 32139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:05.332494 32138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:05.332496 32133 server_base.cc:1034] running on GCE node
W20260924 07:18:05.332459 32141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:05.332844 32133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:05.333130 32133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:05.334267 32133 hybrid_clock.cc:648] HybridClock initialized: now 1790234285334255 us; error 25 us; skew 500 ppm
Sep 24 07:18:05 dist-test-slave-7q7m krb5kdc[32120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234285, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:05.336495 32133 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 07:18:05.336916 32133 webserver.cc:460] Webserver started at http://127.0.0.1:35477/ using document root <none> and password file <none>
I20260924 07:18:05.337119 32133 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:05.337167 32133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:05.337286 32133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:05.338284 32133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "9b2d3390f1b04cdca5721085d33337cf"
format_stamp: "Formatted at 2026-09-24 07:18:05 on dist-test-slave-7q7m"
server_key: "f664205275af46e3d32df1299cd6d740"
server_key_iv: "155d9f8d498c3b06076c2aa8b565ece6"
server_key_version: "encryptionkey@0"
I20260924 07:18:05.338605 32133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "9b2d3390f1b04cdca5721085d33337cf"
format_stamp: "Formatted at 2026-09-24 07:18:05 on dist-test-slave-7q7m"
server_key: "f664205275af46e3d32df1299cd6d740"
server_key_iv: "155d9f8d498c3b06076c2aa8b565ece6"
server_key_version: "encryptionkey@0"
I20260924 07:18:05.339763 32133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 07:18:05.340498 32148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:05.340663 32133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:05.340741 32133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "9b2d3390f1b04cdca5721085d33337cf"
format_stamp: "Formatted at 2026-09-24 07:18:05 on dist-test-slave-7q7m"
server_key: "f664205275af46e3d32df1299cd6d740"
server_key_iv: "155d9f8d498c3b06076c2aa8b565ece6"
server_key_version: "encryptionkey@0"
I20260924 07:18:05.340806 32133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:05.343880 32133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:05.345564 32133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:05.345685 32133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:05.349255 32133 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37741
I20260924 07:18:05.349283 32200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37741 every 8 connection(s)
I20260924 07:18:05.349565 32133 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:18:05.350405 32201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:05.352561 32201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf: Bootstrap starting.
I20260924 07:18:05.352650 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 32133
I20260924 07:18:05.352761 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:18:05.352946 10763 external_mini_cluster.cc:1428] Setting key dc4e0a785f856cc9f907db03b6fcfd6a
I20260924 07:18:05.353168 32201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:05.353477 32201 log.cc:824] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:05.354151 32201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf: No bootstrap required, opened a new log
I20260924 07:18:05.355295 32204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:05.353393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.122:33305 (local address 127.0.0.1:37741)
0924 07:18:05.353574 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:05.353584 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:05.353617 (+ 33us) server_negotiation.cc:407] Connection header received
0924 07:18:05.354141 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:05.354149 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:05.354182 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:05.354326 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:05.354894 (+ 568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:05.354898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:05.354898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:05.354902 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:05.354941 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:05.354954 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:05.355038 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:05.355066 (+ 28us) server_negotiation.cc:299] Negotiation successful
0924 07:18:05.355116 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260924 07:18:05.356182 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:18:05.356328 32201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2d3390f1b04cdca5721085d33337cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37741 } }
I20260924 07:18:05.356477 32201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:18:05.356511 32201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b2d3390f1b04cdca5721085d33337cf, State: Initialized, Role: FOLLOWER
I20260924 07:18:05.356631 32201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2d3390f1b04cdca5721085d33337cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37741 } }
I20260924 07:18:05.356714 32201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:18:05.356747 32201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:18:05.356791 32201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:05.357411 32201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2d3390f1b04cdca5721085d33337cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37741 } }
I20260924 07:18:05.357514 32201 leader_election.cc:302] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b2d3390f1b04cdca5721085d33337cf; no voters:
I20260924 07:18:05.357707 32201 leader_election.cc:288] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:18:05.357730 32211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:18:05.357861 32211 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [term 1 LEADER]: Becoming Leader. State: Replica: 9b2d3390f1b04cdca5721085d33337cf, State: Running, Role: LEADER
I20260924 07:18:05.357939 32201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:05.357987 32211 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2d3390f1b04cdca5721085d33337cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37741 } }
I20260924 07:18:05.358580 32211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b2d3390f1b04cdca5721085d33337cf. Latest consensus state: current_term: 1 leader_uuid: "9b2d3390f1b04cdca5721085d33337cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2d3390f1b04cdca5721085d33337cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37741 } } }
I20260924 07:18:05.358590 32213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b2d3390f1b04cdca5721085d33337cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2d3390f1b04cdca5721085d33337cf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37741 } } }
I20260924 07:18:05.358735 32213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:05.358953 32204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:05.357587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.122:49759 (local address 127.0.0.1:37741)
0924 07:18:05.357647 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:05.357648 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:05.357830 (+ 182us) server_negotiation.cc:407] Connection header received
0924 07:18:05.357850 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:05.357850 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:05.357855 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:05.357893 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:05.358443 (+ 550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:05.358444 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:05.358445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:05.358448 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:05.358460 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:05.358465 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:05.358514 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:05.358675 (+ 161us) server_negotiation.cc:299] Negotiation successful
0924 07:18:05.358687 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 07:18:05.358723 32211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf [sys.catalog]: This master's current role is: LEADER
W20260924 07:18:05.359572 32226 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:18:05.359625 32226 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:18:05.359687 32221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:18:05.359860 32221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:18:05.361233 32221 catalog_manager.cc:1347] Generated new cluster ID: 8dde28df47d2493caac4dd0fa9aea851
I20260924 07:18:05.361284 32221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:18:05.366328 32221 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:18:05.366462 32221 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:18:05.370502 32221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b2d3390f1b04cdca5721085d33337cf: Generated new TSK 0
I20260924 07:18:05.370644 32221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:18:05.376870 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 32133
2026-09-24T07:18:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1177 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:18:05 dist-test-slave-7q7m krb5kdc[32232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:18:05 dist-test-slave-7q7m krb5kdc[32232](info): set up 2 sockets
Sep 24 07:18:05 dist-test-slave-7q7m krb5kdc[32232](info): commencing operation
krb5kdc: starting...
W20260924 07:18:06.323786 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:18:06 dist-test-slave-7q7m krb5kdc[32232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:18:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:18:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:18:06.425093 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:33187
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:44569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:33187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:18:06.502445 32245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:06.502628 32245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:06.502650 32245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:06.503708 32245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:06.503739 32245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:06.503752 32245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:06.503763 32245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:06.503773 32245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:06.505177 32245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:33187
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:33187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:06.505350 32245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:06.505498 32245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:06.509886 32253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:06.509996 32245 server_base.cc:1034] running on GCE node
W20260924 07:18:06.509851 32250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:06.509862 32251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:06.510278 32245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:06.510528 32245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:06.511662 32245 hybrid_clock.cc:648] HybridClock initialized: now 1790234286511648 us; error 27 us; skew 500 ppm
Sep 24 07:18:06 dist-test-slave-7q7m krb5kdc[32232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234286, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:06.513898 32245 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:18:06.514274 32245 webserver.cc:460] Webserver started at http://127.10.130.254:43005/ using document root <none> and password file <none>
I20260924 07:18:06.514461 32245 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:06.514505 32245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:06.514628 32245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:06.515548 32245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "c730210083f441deb13ff00308d113bc"
format_stamp: "Formatted at 2026-09-24 07:18:06 on dist-test-slave-7q7m"
server_key: "4974e86e2e4068bcde7a37a5851582cc"
server_key_iv: "3184afc16d294cfcde0c2f8a245ed30f"
server_key_version: "encryptionkey@0"
I20260924 07:18:06.515847 32245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "c730210083f441deb13ff00308d113bc"
format_stamp: "Formatted at 2026-09-24 07:18:06 on dist-test-slave-7q7m"
server_key: "4974e86e2e4068bcde7a37a5851582cc"
server_key_iv: "3184afc16d294cfcde0c2f8a245ed30f"
server_key_version: "encryptionkey@0"
I20260924 07:18:06.517002 32245 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 07:18:06.517622 32260 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:06.517763 32245 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:18:06.517839 32245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "c730210083f441deb13ff00308d113bc"
format_stamp: "Formatted at 2026-09-24 07:18:06 on dist-test-slave-7q7m"
server_key: "4974e86e2e4068bcde7a37a5851582cc"
server_key_iv: "3184afc16d294cfcde0c2f8a245ed30f"
server_key_version: "encryptionkey@0"
I20260924 07:18:06.517899 32245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:06.530066 32245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:06.531968 32245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:06.532137 32245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:06.535633 32245 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:33187
I20260924 07:18:06.535670 32312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:33187 every 8 connection(s)
I20260924 07:18:06.535955 32245 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:18:06.536760 32313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:06.538846 32313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc: Bootstrap starting.
I20260924 07:18:06.539232 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 32245
I20260924 07:18:06.539345 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:18:06.539460 32313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:06.539535 10763 external_mini_cluster.cc:1428] Setting key 635ec244046a4296f4501d8faf3fa8e6
I20260924 07:18:06.539747 32313 log.cc:824] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:06.540455 32313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc: No bootstrap required, opened a new log
I20260924 07:18:06.542651 32313 raft_consensus.cc:348] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c730210083f441deb13ff00308d113bc" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33187 } }
I20260924 07:18:06.542809 32313 raft_consensus.cc:374] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:18:06.542840 32313 raft_consensus.cc:729] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c730210083f441deb13ff00308d113bc, State: Initialized, Role: FOLLOWER
I20260924 07:18:06.542961 32313 consensus_queue.cc:260] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c730210083f441deb13ff00308d113bc" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33187 } }
I20260924 07:18:06.543037 32313 raft_consensus.cc:388] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:18:06.543061 32313 raft_consensus.cc:482] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:18:06.543104 32313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:06.543612 32313 raft_consensus.cc:504] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c730210083f441deb13ff00308d113bc" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33187 } }
I20260924 07:18:06.543710 32313 leader_election.cc:302] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c730210083f441deb13ff00308d113bc; no voters:
I20260924 07:18:06.543854 32313 leader_election.cc:288] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:18:06.543891 32318 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:18:06.544046 32318 raft_consensus.cc:686] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [term 1 LEADER]: Becoming Leader. State: Replica: c730210083f441deb13ff00308d113bc, State: Running, Role: LEADER
I20260924 07:18:06.544107 32313 sys_catalog.cc:564] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:06.544149 32318 consensus_queue.cc:237] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c730210083f441deb13ff00308d113bc" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33187 } }
I20260924 07:18:06.544502 32319 sys_catalog.cc:455] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c730210083f441deb13ff00308d113bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c730210083f441deb13ff00308d113bc" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33187 } } }
I20260924 07:18:06.544572 32319 sys_catalog.cc:458] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:06.544772 32320 sys_catalog.cc:455] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c730210083f441deb13ff00308d113bc. Latest consensus state: current_term: 1 leader_uuid: "c730210083f441deb13ff00308d113bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c730210083f441deb13ff00308d113bc" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 33187 } } }
I20260924 07:18:06.544998 32320 sys_catalog.cc:458] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:06.545149 32322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:18:06.545553 32322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 07:18:06 dist-test-slave-7q7m krb5kdc[32232](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:18:06.547128 32322 catalog_manager.cc:1347] Generated new cluster ID: cd0e11e3cea44363a7ed2ccaa2c90f3b
I20260924 07:18:06.547185 32322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:18:06.550024 32317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:06.540008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57423 (local address 127.10.130.254:33187)
0924 07:18:06.540154 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:06.540163 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:06.540195 (+ 32us) server_negotiation.cc:407] Connection header received
0924 07:18:06.540733 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:06.540741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:06.540799 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:06.540949 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:06.541477 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:06.542833 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:06.543624 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:06.544249 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:06.544664 (+ 415us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:06.546979 (+ 2315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:06.546982 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:06.546983 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:06.546994 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:06.548605 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:06.548944 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:06.548947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:06.548947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:06.549029 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:06.549233 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:06.549234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:06.549234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:06.549448 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:06.549597 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:06.549733 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 07:18:06.549824 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260924 07:18:06.550891 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:18:06.554989 32322 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:18:06.555603 32322 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:18:06.559023 32317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:06.552336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37167 (local address 127.10.130.254:33187)
0924 07:18:06.552370 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:06.552371 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:06.552492 (+ 121us) server_negotiation.cc:407] Connection header received
0924 07:18:06.552550 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:06.552550 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:06.552566 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:06.552608 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:06.553187 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:06.553909 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:06.554890 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:06.555162 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:06.555230 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:06.556533 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:06.556535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:06.556535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:06.556541 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:06.557943 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:06.558325 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:06.558325 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:06.558326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:06.558376 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:06.558577 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:06.558577 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:06.558578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:06.558676 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:06.558757 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:06.558889 (+ 132us) server_negotiation.cc:299] Negotiation successful
0924 07:18:06.558942 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:18:06.562436 32322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c730210083f441deb13ff00308d113bc: Generated new TSK 0
I20260924 07:18:06.562580 32322 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:18:06.564833 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 32245
W20260924 07:18:06.568709 32230 connection.cc:535] client connection to 127.10.130.254:33187 recv error: Network error: failed to read from TLS socket (remote: 127.10.130.254:33187): Connection reset by peer (error 104)
2026-09-24T07:18:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1187 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:18:06 dist-test-slave-7q7m krb5kdc[32344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:18:06 dist-test-slave-7q7m krb5kdc[32344](info): set up 2 sockets
Sep 24 07:18:06 dist-test-slave-7q7m krb5kdc[32344](info): commencing operation
krb5kdc: starting...
W20260924 07:18:07.503734 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:18:07 dist-test-slave-7q7m krb5kdc[32344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:18:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:18:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:18:07.607843 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:37465
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:44183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:37465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:18:07.683219 32357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:07.683403 32357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:07.683422 32357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:07.684535 32357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:07.684577 32357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:07.684592 32357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:07.684616 32357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:07.684638 32357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:07.686029 32357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:44183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:37465
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:37465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:07.686226 32357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:07.686411 32357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:07.690653 32365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:07.690673 32362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:07.690712 32363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:07.690922 32357 server_base.cc:1034] running on GCE node
I20260924 07:18:07.691131 32357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:07.691376 32357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:07.692505 32357 hybrid_clock.cc:648] HybridClock initialized: now 1790234287692490 us; error 26 us; skew 500 ppm
Sep 24 07:18:07 dist-test-slave-7q7m krb5kdc[32344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234287, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:07.697758 32357 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:18:07.698127 32357 webserver.cc:460] Webserver started at http://127.10.130.254:36245/ using document root <none> and password file <none>
I20260924 07:18:07.698299 32357 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:07.698336 32357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:07.698417 32357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:07.699307 32357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "03980ad65f0e44229fca7585d2d747da"
format_stamp: "Formatted at 2026-09-24 07:18:07 on dist-test-slave-7q7m"
server_key: "08e89819d1a3810ccfef1a56509cd860"
server_key_iv: "21f7c94dde49c365bc6255b9b7347de1"
server_key_version: "encryptionkey@0"
I20260924 07:18:07.699565 32357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "03980ad65f0e44229fca7585d2d747da"
format_stamp: "Formatted at 2026-09-24 07:18:07 on dist-test-slave-7q7m"
server_key: "08e89819d1a3810ccfef1a56509cd860"
server_key_iv: "21f7c94dde49c365bc6255b9b7347de1"
server_key_version: "encryptionkey@0"
I20260924 07:18:07.700701 32357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 07:18:07.701313 32372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:07.701462 32357 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:18:07.701537 32357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "03980ad65f0e44229fca7585d2d747da"
format_stamp: "Formatted at 2026-09-24 07:18:07 on dist-test-slave-7q7m"
server_key: "08e89819d1a3810ccfef1a56509cd860"
server_key_iv: "21f7c94dde49c365bc6255b9b7347de1"
server_key_version: "encryptionkey@0"
I20260924 07:18:07.701586 32357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:07.719079 32357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:07.720906 32357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:07.720999 32357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:07.724325 32357 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:37465
I20260924 07:18:07.724357 32424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:37465 every 8 connection(s)
I20260924 07:18:07.724623 32357 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:18:07.725353 32425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:07.727375 32425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da: Bootstrap starting.
I20260924 07:18:07.727865 32425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:07.728147 32425 log.cc:824] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:07.728758 32425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da: No bootstrap required, opened a new log
I20260924 07:18:07.730245 32425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03980ad65f0e44229fca7585d2d747da" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37465 } }
I20260924 07:18:07.730369 32425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:18:07.730399 32425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03980ad65f0e44229fca7585d2d747da, State: Initialized, Role: FOLLOWER
I20260924 07:18:07.730518 32425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03980ad65f0e44229fca7585d2d747da" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37465 } }
I20260924 07:18:07.730592 32425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:18:07.730623 32425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:18:07.730670 32425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:07.731396 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 32357
I20260924 07:18:07.731417 32425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03980ad65f0e44229fca7585d2d747da" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37465 } }
I20260924 07:18:07.731482 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:18:07.731503 32425 leader_election.cc:302] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03980ad65f0e44229fca7585d2d747da; no voters:
I20260924 07:18:07.731648 32425 leader_election.cc:288] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:18:07.731664 10763 external_mini_cluster.cc:1428] Setting key 22c2b233fb89ab26e5c5307c7ab6f24a
I20260924 07:18:07.731670 32428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:18:07.731774 32428 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [term 1 LEADER]: Becoming Leader. State: Replica: 03980ad65f0e44229fca7585d2d747da, State: Running, Role: LEADER
I20260924 07:18:07.731882 32425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:07.731904 32428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03980ad65f0e44229fca7585d2d747da" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37465 } }
I20260924 07:18:07.732298 32429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03980ad65f0e44229fca7585d2d747da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03980ad65f0e44229fca7585d2d747da" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37465 } } }
I20260924 07:18:07.732368 32429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:07.732542 32430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03980ad65f0e44229fca7585d2d747da. Latest consensus state: current_term: 1 leader_uuid: "03980ad65f0e44229fca7585d2d747da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03980ad65f0e44229fca7585d2d747da" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 37465 } } }
I20260924 07:18:07.732853 32434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:18:07.733417 32434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:18:07.733609 32430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:07.734927 32434 catalog_manager.cc:1347] Generated new cluster ID: f70533546cfa4c1f9376110f8b5494b6
I20260924 07:18:07.734979 32434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:18:07.737010 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:07.732096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44667 (local address 127.10.130.254:37465)
0924 07:18:07.732254 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:07.732263 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:07.732437 (+ 174us) server_negotiation.cc:407] Connection header received
0924 07:18:07.732637 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:07.732644 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:07.732698 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:07.732800 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:07.733176 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:07.734522 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:07.735365 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:07.735778 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:07.735876 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:07.736469 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:07.736473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:07.736473 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:07.736633 (+ 160us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:07.736661 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:07.736671 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:07.736737 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:07.736825 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 07:18:07.736852 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":72,"threads_started":1}
I20260924 07:18:07.737775 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:18:07.742808 32432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:07.740255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58989 (local address 127.10.130.254:37465)
0924 07:18:07.740285 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:07.740287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:07.740431 (+ 144us) server_negotiation.cc:407] Connection header received
0924 07:18:07.740482 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:07.740483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:07.740493 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:07.740543 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:07.740657 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:07.741107 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:07.741781 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:07.741989 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:07.742039 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:07.742594 (+ 555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:07.742595 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:07.742596 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:07.742600 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:07.742612 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:07.742616 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:07.742664 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:07.742729 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 07:18:07.742739 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260924 07:18:07.746351 32434 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:18:07.746968 32434 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:18:07.752069 32434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03980ad65f0e44229fca7585d2d747da: Generated new TSK 0
I20260924 07:18:07.752223 32434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:18:07.757733 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 32357
W20260924 07:18:07.761168 32342 connection.cc:535] client connection to 127.10.130.254:37465 recv error: Network error: failed to read from TLS socket (remote: 127.10.130.254:37465): Connection reset by peer (error 104)
2026-09-24T07:18:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1192 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:18:07 dist-test-slave-7q7m krb5kdc[32456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:18:07 dist-test-slave-7q7m krb5kdc[32456](info): set up 2 sockets
Sep 24 07:18:07 dist-test-slave-7q7m krb5kdc[32456](info): commencing operation
krb5kdc: starting...
W20260924 07:18:08.703815 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.930s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:18:08 dist-test-slave-7q7m krb5kdc[32456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:18:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:18:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:18:08.806532 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:44239
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:44239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:18:08.881920 32469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:08.882094 32469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:08.882113 32469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:08.883194 32469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:08.883234 32469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:08.883246 32469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:08.883257 32469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:08.883267 32469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:08.884671 32469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:44239
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:44239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:08.884856 32469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:08.885046 32469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:08.889294 32474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:08.889304 32475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:08.889304 32477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:08.889370 32469 server_base.cc:1034] running on GCE node
I20260924 07:18:08.889732 32469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:08.889981 32469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:08.891108 32469 hybrid_clock.cc:648] HybridClock initialized: now 1790234288891093 us; error 26 us; skew 500 ppm
Sep 24 07:18:08 dist-test-slave-7q7m krb5kdc[32456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234288, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:08.893332 32469 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:18:08.893724 32469 webserver.cc:460] Webserver started at http://127.10.130.254:39965/ using document root <none> and password file <none>
I20260924 07:18:08.893919 32469 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:08.893965 32469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:08.894079 32469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:08.895013 32469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "4bbe78727ef54c66a9c7655b73a5d07c"
format_stamp: "Formatted at 2026-09-24 07:18:08 on dist-test-slave-7q7m"
server_key: "0e8f5da960f7ff8ae4133ae32a950bdc"
server_key_iv: "ba05b44173ce2b796d133ebe25f778c5"
server_key_version: "encryptionkey@0"
I20260924 07:18:08.895319 32469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "4bbe78727ef54c66a9c7655b73a5d07c"
format_stamp: "Formatted at 2026-09-24 07:18:08 on dist-test-slave-7q7m"
server_key: "0e8f5da960f7ff8ae4133ae32a950bdc"
server_key_iv: "ba05b44173ce2b796d133ebe25f778c5"
server_key_version: "encryptionkey@0"
I20260924 07:18:08.896516 32469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:18:08.897153 32484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:08.897289 32469 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:18:08.897375 32469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "4bbe78727ef54c66a9c7655b73a5d07c"
format_stamp: "Formatted at 2026-09-24 07:18:08 on dist-test-slave-7q7m"
server_key: "0e8f5da960f7ff8ae4133ae32a950bdc"
server_key_iv: "ba05b44173ce2b796d133ebe25f778c5"
server_key_version: "encryptionkey@0"
I20260924 07:18:08.897429 32469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:08.900840 32469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:08.902797 32469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:08.902920 32469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:08.906427 32469 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:44239
I20260924 07:18:08.906459 32536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:44239 every 8 connection(s)
I20260924 07:18:08.906733 32469 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:18:08.907543 32537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:08.909530 32537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c: Bootstrap starting.
I20260924 07:18:08.909847 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 32469
I20260924 07:18:08.909934 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:18:08.910058 32537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:08.910091 10763 external_mini_cluster.cc:1428] Setting key 24a577834addd5a0ce3910c900bf21f6
I20260924 07:18:08.910333 32537 log.cc:824] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:08.911185 32537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c: No bootstrap required, opened a new log
I20260924 07:18:08.912295 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:08.910502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46327 (local address 127.10.130.254:44239)
0924 07:18:08.910648 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:08.910657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:08.910690 (+ 33us) server_negotiation.cc:407] Connection header received
0924 07:18:08.911188 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:08.911196 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:08.911228 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:08.911372 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:08.911876 (+ 504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:08.911879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:08.911879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:08.911883 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:08.911922 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:08.911935 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:08.912042 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:08.912071 (+ 29us) server_negotiation.cc:299] Negotiation successful
0924 07:18:08.912107 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
I20260924 07:18:08.913192 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:18:08.913369 32537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbe78727ef54c66a9c7655b73a5d07c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44239 } }
I20260924 07:18:08.913487 32537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:18:08.913518 32537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bbe78727ef54c66a9c7655b73a5d07c, State: Initialized, Role: FOLLOWER
I20260924 07:18:08.913625 32537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbe78727ef54c66a9c7655b73a5d07c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44239 } }
I20260924 07:18:08.913709 32537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:18:08.913739 32537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:18:08.913784 32537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:08.914281 32537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbe78727ef54c66a9c7655b73a5d07c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44239 } }
I20260924 07:18:08.914371 32537 leader_election.cc:302] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bbe78727ef54c66a9c7655b73a5d07c; no voters:
I20260924 07:18:08.914529 32537 leader_election.cc:288] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:18:08.914554 32547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:18:08.914767 32547 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [term 1 LEADER]: Becoming Leader. State: Replica: 4bbe78727ef54c66a9c7655b73a5d07c, State: Running, Role: LEADER
I20260924 07:18:08.914819 32537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:08.914898 32547 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbe78727ef54c66a9c7655b73a5d07c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44239 } }
I20260924 07:18:08.915242 32549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bbe78727ef54c66a9c7655b73a5d07c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbe78727ef54c66a9c7655b73a5d07c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44239 } } }
I20260924 07:18:08.915315 32549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:08.915530 32540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:08.914430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34177 (local address 127.10.130.254:44239)
0924 07:18:08.914461 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:08.914463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:08.914584 (+ 121us) server_negotiation.cc:407] Connection header received
0924 07:18:08.914635 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:08.914636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:08.914641 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:08.914696 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:08.915220 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:08.915221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:08.915222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:08.915225 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:08.915236 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:08.915241 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:08.915284 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:08.915463 (+ 179us) server_negotiation.cc:299] Negotiation successful
0924 07:18:08.915475 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:18:08.915764 32548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bbe78727ef54c66a9c7655b73a5d07c. Latest consensus state: current_term: 1 leader_uuid: "4bbe78727ef54c66a9c7655b73a5d07c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bbe78727ef54c66a9c7655b73a5d07c" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44239 } } }
I20260924 07:18:08.915879 32548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:08.916153 32553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:18:08.916538 32553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:18:08.917764 32553 catalog_manager.cc:1347] Generated new cluster ID: b37b4d216ee64978b4306f675e6aab61
I20260924 07:18:08.917814 32553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:18:08.925748 32553 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:18:08.925881 32553 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:18:08.932842 32553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bbe78727ef54c66a9c7655b73a5d07c: Generated new TSK 0
I20260924 07:18:08.932986 32553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:18:08.941071 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 32469
2026-09-24T07:18:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1183 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:18:08 dist-test-slave-7q7m krb5kdc[32568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:18:08 dist-test-slave-7q7m krb5kdc[32568](info): set up 2 sockets
Sep 24 07:18:08 dist-test-slave-7q7m krb5kdc[32568](info): commencing operation
krb5kdc: starting...
W20260924 07:18:09.891656 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.934s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:18:09 dist-test-slave-7q7m krb5kdc[32568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:18:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:18:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:18:09.996425 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:36171
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:36643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:36171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:18:10.073602 32581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:10.073782 32581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:10.073802 32581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:10.074883 32581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:10.074920 32581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:10.074934 32581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:10.074944 32581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:10.074957 32581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:10.076391 32581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:36643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:36171
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:36171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:10.076596 32581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:10.076784 32581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:10.081043 32587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:10.081149 32581 server_base.cc:1034] running on GCE node
W20260924 07:18:10.081115 32589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:10.081049 32586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:10.081478 32581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:10.081754 32581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:10.082897 32581 hybrid_clock.cc:648] HybridClock initialized: now 1790234290082875 us; error 33 us; skew 500 ppm
Sep 24 07:18:10 dist-test-slave-7q7m krb5kdc[32568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234290, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:10.085170 32581 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:18:10.085564 32581 webserver.cc:460] Webserver started at http://127.10.130.254:39087/ using document root <none> and password file <none>
I20260924 07:18:10.085747 32581 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:10.085795 32581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:10.085922 32581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:10.086915 32581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "11e0e0121ec343f5b3c5048c1b0f3d74"
format_stamp: "Formatted at 2026-09-24 07:18:10 on dist-test-slave-7q7m"
server_key: "a0253534193bc25b2f446be0eb5db352"
server_key_iv: "f6a387fd0feeb657f09d8c3a564fa78d"
server_key_version: "encryptionkey@0"
I20260924 07:18:10.087256 32581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "11e0e0121ec343f5b3c5048c1b0f3d74"
format_stamp: "Formatted at 2026-09-24 07:18:10 on dist-test-slave-7q7m"
server_key: "a0253534193bc25b2f446be0eb5db352"
server_key_iv: "f6a387fd0feeb657f09d8c3a564fa78d"
server_key_version: "encryptionkey@0"
I20260924 07:18:10.088557 32581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:18:10.089200 32596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:10.089344 32581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:18:10.089428 32581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "11e0e0121ec343f5b3c5048c1b0f3d74"
format_stamp: "Formatted at 2026-09-24 07:18:10 on dist-test-slave-7q7m"
server_key: "a0253534193bc25b2f446be0eb5db352"
server_key_iv: "f6a387fd0feeb657f09d8c3a564fa78d"
server_key_version: "encryptionkey@0"
I20260924 07:18:10.089489 32581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:10.108837 32581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:10.110589 32581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:10.110728 32581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:10.114596 32581 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:36171
I20260924 07:18:10.114638 32648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:36171 every 8 connection(s)
I20260924 07:18:10.114957 32581 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:18:10.115752 32649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:10.117924 32649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74: Bootstrap starting.
I20260924 07:18:10.118451 32649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:10.118697 32649 log.cc:824] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:10.119330 32649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74: No bootstrap required, opened a new log
I20260924 07:18:10.120272 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 32581
I20260924 07:18:10.120368 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:18:10.120558 10763 external_mini_cluster.cc:1428] Setting key 8a0f1f1e3311e871056e41cac1779978
I20260924 07:18:10.121073 32649 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0e0121ec343f5b3c5048c1b0f3d74" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36171 } }
I20260924 07:18:10.121260 32649 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:18:10.121309 32649 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11e0e0121ec343f5b3c5048c1b0f3d74, State: Initialized, Role: FOLLOWER
I20260924 07:18:10.121426 32649 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0e0121ec343f5b3c5048c1b0f3d74" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36171 } }
I20260924 07:18:10.121500 32649 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:18:10.121528 32649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:18:10.121573 32649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:10.122218 32649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0e0121ec343f5b3c5048c1b0f3d74" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36171 } }
I20260924 07:18:10.122325 32649 leader_election.cc:302] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11e0e0121ec343f5b3c5048c1b0f3d74; no voters:
I20260924 07:18:10.122474 32649 leader_election.cc:288] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:18:10.122520 32654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:18:10.122714 32654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [term 1 LEADER]: Becoming Leader. State: Replica: 11e0e0121ec343f5b3c5048c1b0f3d74, State: Running, Role: LEADER
I20260924 07:18:10.122740 32649 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:10.122825 32654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0e0121ec343f5b3c5048c1b0f3d74" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36171 } }
I20260924 07:18:10.123128 32654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11e0e0121ec343f5b3c5048c1b0f3d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0e0121ec343f5b3c5048c1b0f3d74" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36171 } } }
I20260924 07:18:10.123243 32654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:10.123400 32654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11e0e0121ec343f5b3c5048c1b0f3d74. Latest consensus state: current_term: 1 leader_uuid: "11e0e0121ec343f5b3c5048c1b0f3d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11e0e0121ec343f5b3c5048c1b0f3d74" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 36171 } } }
I20260924 07:18:10.123513 32654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74 [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:10.123821 32663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:18:10.124331 32663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:18:10.127450 32663 catalog_manager.cc:1347] Generated new cluster ID: 2807d7cb6e864cb59aa5b075dacfafc2
I20260924 07:18:10.127508 32663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 07:18:10 dist-test-slave-7q7m krb5kdc[32568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:18:10.131079 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:10.121041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43793 (local address 127.10.130.254:36171)
0924 07:18:10.121238 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:10.121260 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:10.121292 (+ 32us) server_negotiation.cc:407] Connection header received
0924 07:18:10.121409 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:10.121416 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:10.121476 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:10.121624 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:10.122717 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:10.124100 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:10.125129 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:10.125661 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:10.125757 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:10.128246 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:10.128251 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:10.128251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:10.128262 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:10.129770 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:10.130068 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:10.130071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:10.130071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:10.130161 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:10.130329 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:10.130330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:10.130330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:10.130554 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:10.130715 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:10.130778 (+ 63us) server_negotiation.cc:299] Negotiation successful
0924 07:18:10.130902 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260924 07:18:10.131867 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:18:10.138327 32653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:10.133096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37087 (local address 127.10.130.254:36171)
0924 07:18:10.133130 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:10.133131 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:10.133237 (+ 106us) server_negotiation.cc:407] Connection header received
0924 07:18:10.133287 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:10.133288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:10.133296 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:10.133339 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:10.133778 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:10.134238 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:10.134887 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:10.135105 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:10.135157 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:10.136217 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:10.136218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:10.136218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:10.136224 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:10.137417 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:10.137707 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:10.137708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:10.137708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:10.137758 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:10.137911 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:10.137912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:10.137913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:10.138009 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:10.138094 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:10.138181 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 07:18:10.138247 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:18:10.141240 32663 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:18:10.141945 32663 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:18:10.154491 32663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11e0e0121ec343f5b3c5048c1b0f3d74: Generated new TSK 0
I20260924 07:18:10.154707 32663 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:18:10.165705 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 32581
2026-09-24T07:18:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1225 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:18:10 dist-test-slave-7q7m krb5kdc[32680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:18:10 dist-test-slave-7q7m krb5kdc[32680](info): set up 2 sockets
Sep 24 07:18:10 dist-test-slave-7q7m krb5kdc[32680](info): commencing operation
krb5kdc: starting...
W20260924 07:18:11.099709 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.917s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:18:11 dist-test-slave-7q7m krb5kdc[32680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:18:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:18:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:18:11.204530 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:44401
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:44401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:18:11.280529 32693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:11.280720 32693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:11.280741 32693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:11.281827 32693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:11.281878 32693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:11.281890 32693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:11.281901 32693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:11.281911 32693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:11.283303 32693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:44401
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:44401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:11.283481 32693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:11.283643 32693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:11.288002 32698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:11.288074 32699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:11.288080 32693 server_base.cc:1034] running on GCE node
W20260924 07:18:11.288020 32701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:11.288379 32693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:11.288633 32693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:11.289764 32693 hybrid_clock.cc:648] HybridClock initialized: now 1790234291289744 us; error 34 us; skew 500 ppm
Sep 24 07:18:11 dist-test-slave-7q7m krb5kdc[32680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234291, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:11.292068 32693 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:18:11.292491 32693 webserver.cc:460] Webserver started at http://127.10.130.254:44445/ using document root <none> and password file <none>
I20260924 07:18:11.292688 32693 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:11.292734 32693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:11.292853 32693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:11.293838 32693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "0f69b273ba0b44d7b314ec214fe2c3dc"
format_stamp: "Formatted at 2026-09-24 07:18:11 on dist-test-slave-7q7m"
server_key: "4f73f72e1b2fbdcbd0d6fdca79fd0a8c"
server_key_iv: "a1f0818a41eb32db8a39250c8f5edb50"
server_key_version: "encryptionkey@0"
I20260924 07:18:11.294152 32693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "0f69b273ba0b44d7b314ec214fe2c3dc"
format_stamp: "Formatted at 2026-09-24 07:18:11 on dist-test-slave-7q7m"
server_key: "4f73f72e1b2fbdcbd0d6fdca79fd0a8c"
server_key_iv: "a1f0818a41eb32db8a39250c8f5edb50"
server_key_version: "encryptionkey@0"
I20260924 07:18:11.295339 32693 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:18:11.296013 32708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:11.296159 32693 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:11.296243 32693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "0f69b273ba0b44d7b314ec214fe2c3dc"
format_stamp: "Formatted at 2026-09-24 07:18:11 on dist-test-slave-7q7m"
server_key: "4f73f72e1b2fbdcbd0d6fdca79fd0a8c"
server_key_iv: "a1f0818a41eb32db8a39250c8f5edb50"
server_key_version: "encryptionkey@0"
I20260924 07:18:11.296308 32693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:11.321334 32693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:11.323421 32693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:11.323551 32693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:11.327196 32693 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:44401
I20260924 07:18:11.327224 32760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:44401 every 8 connection(s)
I20260924 07:18:11.327533 32693 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:18:11.328233 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 32693
I20260924 07:18:11.328330 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:18:11.328336 32761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:11.328496 10763 external_mini_cluster.cc:1428] Setting key 6559dd04310597e1fafcd7e053d720a6
I20260924 07:18:11.331547 32761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc: Bootstrap starting.
I20260924 07:18:11.332152 32761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:11.332435 32761 log.cc:824] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:11.333117 32761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc: No bootstrap required, opened a new log
I20260924 07:18:11.334100 32764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:11.328933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35923 (local address 127.10.130.254:44401)
0924 07:18:11.329133 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:11.329143 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:11.329177 (+ 34us) server_negotiation.cc:407] Connection header received
0924 07:18:11.329729 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:11.329738 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:11.329800 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:11.329947 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:11.330092 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:11.331405 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:11.332152 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:11.332716 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:11.332797 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:11.333449 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:11.333454 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:11.333454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:11.333653 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:11.333691 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:11.333705 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:11.333767 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:11.333822 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 07:18:11.333888 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260924 07:18:11.335062 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:18:11.335295 32761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f69b273ba0b44d7b314ec214fe2c3dc" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44401 } }
I20260924 07:18:11.335421 32761 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:18:11.335441 32761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f69b273ba0b44d7b314ec214fe2c3dc, State: Initialized, Role: FOLLOWER
I20260924 07:18:11.335552 32761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f69b273ba0b44d7b314ec214fe2c3dc" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44401 } }
I20260924 07:18:11.335614 32761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:18:11.335634 32761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:18:11.335680 32761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:11.336262 32761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f69b273ba0b44d7b314ec214fe2c3dc" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44401 } }
I20260924 07:18:11.336431 32761 leader_election.cc:302] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f69b273ba0b44d7b314ec214fe2c3dc; no voters:
I20260924 07:18:11.336560 32761 leader_election.cc:288] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:18:11.336601 303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:18:11.336762 303 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [term 1 LEADER]: Becoming Leader. State: Replica: 0f69b273ba0b44d7b314ec214fe2c3dc, State: Running, Role: LEADER
I20260924 07:18:11.336761 32761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:11.336923 303 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f69b273ba0b44d7b314ec214fe2c3dc" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44401 } }
I20260924 07:18:11.337420 304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f69b273ba0b44d7b314ec214fe2c3dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f69b273ba0b44d7b314ec214fe2c3dc" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44401 } } }
I20260924 07:18:11.337500 304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:11.337844 305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f69b273ba0b44d7b314ec214fe2c3dc. Latest consensus state: current_term: 1 leader_uuid: "0f69b273ba0b44d7b314ec214fe2c3dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f69b273ba0b44d7b314ec214fe2c3dc" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 44401 } } }
I20260924 07:18:11.337929 305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:11.338086 312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:18:11.338482 312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:18:11.339939 312 catalog_manager.cc:1347] Generated new cluster ID: 7d8bc2ed2a3a42ada19a383e6123719c
I20260924 07:18:11.340011 312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:18:11.340372 32764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:11.336370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55207 (local address 127.10.130.254:44401)
0924 07:18:11.336406 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:11.336407 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:11.336944 (+ 537us) server_negotiation.cc:407] Connection header received
0924 07:18:11.337046 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:11.337047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:11.337058 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:11.337085 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:11.337256 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:11.337921 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:11.338818 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:11.339087 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:11.339152 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:11.340027 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:11.340028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:11.340028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:11.340032 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:11.340045 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:11.340049 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:11.340088 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:11.340285 (+ 197us) server_negotiation.cc:299] Negotiation successful
0924 07:18:11.340304 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:18:11.346933 312 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:18:11.347391 312 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:18:11.350239 312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0f69b273ba0b44d7b314ec214fe2c3dc: Generated new TSK 0
I20260924 07:18:11.350369 312 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:18:11.356678 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 32693
2026-09-24T07:18:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1190 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:18:11 dist-test-slave-7q7m krb5kdc[324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:18:11 dist-test-slave-7q7m krb5kdc[324](info): set up 2 sockets
Sep 24 07:18:11 dist-test-slave-7q7m krb5kdc[324](info): commencing operation
krb5kdc: starting...
W20260924 07:18:12.295761 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.922s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:18:12 dist-test-slave-7q7m krb5kdc[324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:18:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:18:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:18:12.398288 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:41539
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:38711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:41539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:18:12.481313 337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:12.481489 337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:12.481518 337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:12.482597 337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:12.482636 337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:12.482656 337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:12.482669 337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:12.482685 337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:12.484120 337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:38711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:41539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:41539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:12.484345 337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:12.484525 337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:12.488660 343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:12.488660 345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:12.488646 342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:12.488866 337 server_base.cc:1034] running on GCE node
I20260924 07:18:12.489096 337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:12.489348 337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:12.490475 337 hybrid_clock.cc:648] HybridClock initialized: now 1790234292490463 us; error 24 us; skew 500 ppm
Sep 24 07:18:12 dist-test-slave-7q7m krb5kdc[324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234292, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:12.492552 337 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:18:12.492949 337 webserver.cc:460] Webserver started at http://127.10.130.254:46399/ using document root <none> and password file <none>
I20260924 07:18:12.493155 337 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:12.493211 337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:12.493326 337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:12.494237 337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "c20c38bd749a482a8919ba61ea119214"
format_stamp: "Formatted at 2026-09-24 07:18:12 on dist-test-slave-7q7m"
server_key: "7752329d0cf4197eecd3cc809dd6ed0c"
server_key_iv: "f2ef13964f25ed0d8e28dee166a4b5dc"
server_key_version: "encryptionkey@0"
I20260924 07:18:12.494534 337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "c20c38bd749a482a8919ba61ea119214"
format_stamp: "Formatted at 2026-09-24 07:18:12 on dist-test-slave-7q7m"
server_key: "7752329d0cf4197eecd3cc809dd6ed0c"
server_key_iv: "f2ef13964f25ed0d8e28dee166a4b5dc"
server_key_version: "encryptionkey@0"
I20260924 07:18:12.495667 337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 07:18:12.496286 352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:12.496428 337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:12.496503 337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "c20c38bd749a482a8919ba61ea119214"
format_stamp: "Formatted at 2026-09-24 07:18:12 on dist-test-slave-7q7m"
server_key: "7752329d0cf4197eecd3cc809dd6ed0c"
server_key_iv: "f2ef13964f25ed0d8e28dee166a4b5dc"
server_key_version: "encryptionkey@0"
I20260924 07:18:12.496559 337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:12.499483 337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:12.501032 337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:12.501147 337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:12.504571 337 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:41539
I20260924 07:18:12.504599 404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:41539 every 8 connection(s)
I20260924 07:18:12.504850 337 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:18:12.505622 405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:12.507458 405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214: Bootstrap starting.
I20260924 07:18:12.507931 405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:12.508214 405 log.cc:824] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:12.508759 405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214: No bootstrap required, opened a new log
I20260924 07:18:12.510195 405 raft_consensus.cc:348] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20c38bd749a482a8919ba61ea119214" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 41539 } }
I20260924 07:18:12.510310 405 raft_consensus.cc:374] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 07:18:12.510339 405 raft_consensus.cc:729] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c20c38bd749a482a8919ba61ea119214, State: Initialized, Role: FOLLOWER
I20260924 07:18:12.510465 405 consensus_queue.cc:260] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20c38bd749a482a8919ba61ea119214" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 41539 } }
I20260924 07:18:12.510533 405 raft_consensus.cc:388] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 07:18:12.510567 405 raft_consensus.cc:482] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 07:18:12.510609 405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:12.511312 405 raft_consensus.cc:504] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20c38bd749a482a8919ba61ea119214" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 41539 } }
I20260924 07:18:12.511396 405 leader_election.cc:302] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c20c38bd749a482a8919ba61ea119214; no voters:
I20260924 07:18:12.511536 405 leader_election.cc:288] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 07:18:12.511548 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 337
I20260924 07:18:12.511574 408 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:18:12.511634 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:18:12.511722 408 raft_consensus.cc:686] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [term 1 LEADER]: Becoming Leader. State: Replica: c20c38bd749a482a8919ba61ea119214, State: Running, Role: LEADER
I20260924 07:18:12.511763 405 sys_catalog.cc:564] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:12.511817 10763 external_mini_cluster.cc:1428] Setting key 5d7818b726de3354c6f9e6aab7fcc726
I20260924 07:18:12.511791 408 consensus_queue.cc:237] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20c38bd749a482a8919ba61ea119214" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 41539 } }
I20260924 07:18:12.512391 410 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c20c38bd749a482a8919ba61ea119214. Latest consensus state: current_term: 1 leader_uuid: "c20c38bd749a482a8919ba61ea119214" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20c38bd749a482a8919ba61ea119214" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 41539 } } }
I20260924 07:18:12.512465 410 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:12.512655 409 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c20c38bd749a482a8919ba61ea119214" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20c38bd749a482a8919ba61ea119214" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 41539 } } }
I20260924 07:18:12.512738 419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:18:12.512790 409 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214 [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:12.513777 419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:18:12.514599 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:12.512259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39451 (local address 127.10.130.254:41539)
0924 07:18:12.512418 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:12.512428 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:12.512820 (+ 392us) server_negotiation.cc:407] Connection header received
0924 07:18:12.513132 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:12.513139 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:12.513167 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:12.513265 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:12.514183 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:12.514186 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:12.514187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:12.514191 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:12.514232 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:12.514245 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:12.514294 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:12.514390 (+ 96us) server_negotiation.cc:299] Negotiation successful
0924 07:18:12.514430 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":65,"threads_started":1}
I20260924 07:18:12.515205 419 catalog_manager.cc:1347] Generated new cluster ID: df0f9e037c4b4b36b804578cc947f9ba
I20260924 07:18:12.515276 419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:18:12.515552 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:18:12.517916 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:12.516855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49507 (local address 127.10.130.254:41539)
0924 07:18:12.516908 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:12.516909 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:12.517110 (+ 201us) server_negotiation.cc:407] Connection header received
0924 07:18:12.517140 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:12.517141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:12.517146 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:12.517211 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:12.517715 (+ 504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:12.517716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:12.517716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 07:18:12.517720 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:12.517734 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 07:18:12.517738 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:12.517790 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:12.517837 (+ 47us) server_negotiation.cc:299] Negotiation successful
0924 07:18:12.517849 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 07:18:12.526790 419 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:18:12.526929 419 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:18:12.535018 419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c20c38bd749a482a8919ba61ea119214: Generated new TSK 0
I20260924 07:18:12.535161 419 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:18:12.544250 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 337
2026-09-24T07:18:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1187 ms)
[----------] 18 tests from AuthTokenIssuingTest (21472 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:18:12 dist-test-slave-7q7m krb5kdc[436](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:18:12 dist-test-slave-7q7m krb5kdc[436](info): set up 2 sockets
Sep 24 07:18:12 dist-test-slave-7q7m krb5kdc[436](info): commencing operation
krb5kdc: starting...
W20260924 07:18:13.483708 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.923s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:18:13 dist-test-slave-7q7m krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:18:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:18:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:18:13.586479 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:42769
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:46869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:42769,127.10.130.253:46637,127.10.130.252:35343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:18:13.662773 449 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 07:18:13.662963 449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:13.662986 449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:13.663010 449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:13.664156 449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:13.664193 449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:13.664212 449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:13.664227 449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:13.664243 449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:13.665666 449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:46869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:42769,127.10.130.253:46637,127.10.130.252:35343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:42769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:13.665887 449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:13.666082 449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:13.670321 454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:13.670322 457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:13.670414 449 server_base.cc:1034] running on GCE node
W20260924 07:18:13.670344 455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:13.670725 449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:13.670976 449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:13.672101 449 hybrid_clock.cc:648] HybridClock initialized: now 1790234293672088 us; error 30 us; skew 500 ppm
Sep 24 07:18:13 dist-test-slave-7q7m krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234293, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:13.677068 449 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:18:13.677464 449 webserver.cc:460] Webserver started at http://127.10.130.254:43825/ using document root <none> and password file <none>
I20260924 07:18:13.677657 449 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:13.677698 449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:13.677811 449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:13.678771 449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "10c09e06fcd14782bca4314762d8535b"
format_stamp: "Formatted at 2026-09-24 07:18:13 on dist-test-slave-7q7m"
server_key: "c401292717355ff24c6e4d08008fd554"
server_key_iv: "15872cb1f66cb282033c710ae4a1aa5e"
server_key_version: "encryptionkey@0"
I20260924 07:18:13.679116 449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "10c09e06fcd14782bca4314762d8535b"
format_stamp: "Formatted at 2026-09-24 07:18:13 on dist-test-slave-7q7m"
server_key: "c401292717355ff24c6e4d08008fd554"
server_key_iv: "15872cb1f66cb282033c710ae4a1aa5e"
server_key_version: "encryptionkey@0"
I20260924 07:18:13.680316 449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:18:13.680967 464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:13.681116 449 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:18:13.681187 449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "10c09e06fcd14782bca4314762d8535b"
format_stamp: "Formatted at 2026-09-24 07:18:13 on dist-test-slave-7q7m"
server_key: "c401292717355ff24c6e4d08008fd554"
server_key_iv: "15872cb1f66cb282033c710ae4a1aa5e"
server_key_version: "encryptionkey@0"
I20260924 07:18:13.681262 449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:13.740751 449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:13.742692 449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:13.742811 449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:13.746484 449 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:42769
I20260924 07:18:13.746521 516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:42769 every 8 connection(s)
I20260924 07:18:13.746845 449 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:18:13.747625 517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:13.749101 517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:18:13.750625 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 449
I20260924 07:18:13.750725 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:18:13.750907 10763 external_mini_cluster.cc:1428] Setting key ee2b030d3d1f75d8664467222aa5ff7e
Sep 24 07:18:13 dist-test-slave-7q7m krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234293, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.130.253@KRBTEST.COM; defaulting to no policy
I20260924 07:18:13.758421 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.749704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:42769 (local address 127.0.0.1:55052)
0924 07:18:13.749920 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.749948 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:13.750372 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:13.750616 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:13.750619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:13.750949 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:13.751172 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:13.751178 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.752257 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.752258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:13.752952 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:13.752955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.753449 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.753450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:13.753524 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:13.753969 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:13.753979 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:13.755515 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:13.757050 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:13.757057 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:13.757059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:13.757300 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:13.757499 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:13.757500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:13.757500 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:13.757598 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:13.757914 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:13.757921 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:13.757995 (+ 74us) client_negotiation.cc:769] Sending connection context
0924 07:18:13.758085 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 07:18:13.758219 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260924 07:18:13.758424 519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.749724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55052 (local address 127.10.130.254:42769)
0924 07:18:13.749866 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:13.749875 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:13.750018 (+ 143us) server_negotiation.cc:407] Connection header received
0924 07:18:13.750488 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:13.750489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:13.750506 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:13.750583 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:13.751263 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.752177 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.753032 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.753369 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.753447 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:13.755597 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:13.755603 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:13.755604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:13.755609 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:13.756967 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:13.757373 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:13.757375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:13.757375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:13.757444 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:13.757654 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:13.757655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:13.757655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:13.757850 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:13.757913 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:13.758116 (+ 203us) server_negotiation.cc:299] Negotiation successful
0924 07:18:13.758263 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260924 07:18:13.759068 517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:18:13.759328 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.759236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:46637 (local address 127.0.0.1:33982)
0924 07:18:13.759271 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.759311 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260924 07:18:13.759440 465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.130.253:46637: connect: Connection refused (error 111)
I20260924 07:18:13.759765 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.759707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:46637 (local address 127.0.0.1:33986)
0924 07:18:13.759729 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.759750 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260924 07:18:13.759846 517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.253:46637: Network error: Client connection negotiation failed: client connection to 127.10.130.253:46637: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.253.keytab.
Entry for principal kudu/127.10.130.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.253.keytab.
I20260924 07:18:13.772215 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.253:46637
--webserver_interface=127.10.130.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.253.keytab
--principal=kudu/127.10.130.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:46869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:42769,127.10.130.253:46637,127.10.130.252:35343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:18:13.809055 517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } attempt: 1
I20260924 07:18:13.809551 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.809396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:46637 (local address 127.0.0.1:33988)
0924 07:18:13.809463 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.809517 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260924 07:18:13.809918 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.809834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:46637 (local address 127.0.0.1:34004)
0924 07:18:13.809872 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.809898 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 07:18:13.810029 517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.253:46637: Network error: Client connection negotiation failed: client connection to 127.10.130.253:46637: connect: Connection refused (error 111)
W20260924 07:18:13.848177 524 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 07:18:13.848356 524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:13.848372 524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:13.848390 524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:13.849491 524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:13.849527 524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:13.849540 524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:13.849551 524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:13.849561 524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:13.850944 524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:46869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:42769,127.10.130.253:46637,127.10.130.252:35343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.253.keytab
--principal=kudu/127.10.130.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.253:46637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.130.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:13.851159 524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:13.851342 524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:13.855538 530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:13.855576 532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:13.855585 529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:13.855609 524 server_base.cc:1034] running on GCE node
I20260924 07:18:13.855867 524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:13.856169 524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:13.857319 524 hybrid_clock.cc:648] HybridClock initialized: now 1790234293857301 us; error 29 us; skew 500 ppm
Sep 24 07:18:13 dist-test-slave-7q7m krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234293, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:13.859570 524 init.cc:377] Logged in from keytab as kudu/127.10.130.253@KRBTEST.COM (short username kudu)
I20260924 07:18:13.859962 524 webserver.cc:460] Webserver started at http://127.10.130.253:42957/ using document root <none> and password file <none>
I20260924 07:18:13.860184 524 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:13.860224 524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:13.860311 524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:13.861290 524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/data/instance:
uuid: "f064fed3299f4107997eba6c3d9b4c7e"
format_stamp: "Formatted at 2026-09-24 07:18:13 on dist-test-slave-7q7m"
server_key: "0edf80853cfa57e2ee6fd4d19423d3a5"
server_key_iv: "de20011c9e5af4fe597a0e3a2fae1be0"
server_key_version: "encryptionkey@0"
I20260924 07:18:13.861567 524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/wal/instance:
uuid: "f064fed3299f4107997eba6c3d9b4c7e"
format_stamp: "Formatted at 2026-09-24 07:18:13 on dist-test-slave-7q7m"
server_key: "0edf80853cfa57e2ee6fd4d19423d3a5"
server_key_iv: "de20011c9e5af4fe597a0e3a2fae1be0"
server_key_version: "encryptionkey@0"
I20260924 07:18:13.862731 524 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 07:18:13.863441 539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:13.863596 524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:18:13.863682 524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/wal
uuid: "f064fed3299f4107997eba6c3d9b4c7e"
format_stamp: "Formatted at 2026-09-24 07:18:13 on dist-test-slave-7q7m"
server_key: "0edf80853cfa57e2ee6fd4d19423d3a5"
server_key_iv: "de20011c9e5af4fe597a0e3a2fae1be0"
server_key_version: "encryptionkey@0"
I20260924 07:18:13.863744 524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:13.878175 517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } attempt: 2
I20260924 07:18:13.878633 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.878500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:46637 (local address 127.0.0.1:34014)
0924 07:18:13.878559 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.878602 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 07:18:13.878968 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.878848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:46637 (local address 127.0.0.1:34020)
0924 07:18:13.878909 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.878940 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:46637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260924 07:18:13.879058 517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.253:46637: Network error: Client connection negotiation failed: client connection to 127.10.130.253:46637: connect: Connection refused (error 111)
I20260924 07:18:13.886221 524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:13.887929 524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:13.888072 524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:13.891532 524 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.253:46637
I20260924 07:18:13.891571 591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.253:46637 every 8 connection(s)
I20260924 07:18:13.891875 524 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/data/info.pb
I20260924 07:18:13.892681 592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:13.894145 592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:18:13.895867 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 524
I20260924 07:18:13.895951 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-1/wal/instance
I20260924 07:18:13.896136 10763 external_mini_cluster.cc:1428] Setting key 24f5aaaf16d07dc8c445fefbbe09f98f
Sep 24 07:18:13 dist-test-slave-7q7m krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234293, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.253@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:18:13.902787 519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.894764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55068 (local address 127.10.130.254:42769)
0924 07:18:13.894819 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:13.894820 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:13.895023 (+ 203us) server_negotiation.cc:407] Connection header received
0924 07:18:13.895517 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:13.895518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:13.895532 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:13.895589 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:13.896265 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.896752 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.897690 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.897900 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.897953 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:13.900205 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:13.900206 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:13.900206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:13.900213 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:13.901457 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:13.901935 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:13.901936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:13.901936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:13.901985 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:13.902232 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:13.902232 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:13.902233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:13.902326 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:13.902369 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:13.902672 (+ 303us) server_negotiation.cc:299] Negotiation successful
0924 07:18:13.902728 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 07:18:13.903046 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.894754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:42769 (local address 127.0.0.1:55068)
0924 07:18:13.894912 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.894947 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:13.895423 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:13.895660 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:13.895665 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:13.895898 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:13.896145 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:13.896151 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.896834 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.896835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:13.897599 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:13.897602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.897970 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.897971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:13.898085 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:13.898596 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:13.898605 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:13.900100 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:13.901552 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:13.901561 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:13.901563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:13.901864 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:13.902045 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:13.902046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:13.902046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:13.902174 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:13.902381 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:13.902387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:13.902546 (+ 159us) client_negotiation.cc:769] Sending connection context
0924 07:18:13.902665 (+ 119us) client_negotiation.cc:240] Negotiation successful
0924 07:18:13.902834 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
WARNING: no policy specified for kudu/127.10.130.252@KRBTEST.COM; defaulting to no policy
I20260924 07:18:13.903584 592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.252@KRBTEST.COM" created.
Sep 24 07:18:13 dist-test-slave-7q7m krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234293, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.253@KRBTEST.COM for kudu/127.10.130.253@KRBTEST.COM
I20260924 07:18:13.911103 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.903780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:46637 (local address 127.0.0.1:34034)
0924 07:18:13.903831 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.903840 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:13.903873 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:13.904124 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:13.904124 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:13.904258 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:13.904316 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:13.904318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.905084 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.905085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:13.905673 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:13.905675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.907037 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.907038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:13.907123 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:13.907428 (+ 305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:13.907433 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:13.908526 (+ 1093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:13.910266 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:13.910266 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:13.910266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:13.910405 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:13.910596 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:13.910596 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:13.910597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:13.910637 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:13.910915 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:13.910916 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:13.910946 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 07:18:13.910989 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 07:18:13.911047 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 07:18:13.911223 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.903829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34034 (local address 127.10.130.253:46637)
0924 07:18:13.903966 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:13.903970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:13.904018 (+ 48us) server_negotiation.cc:407] Connection header received
0924 07:18:13.904039 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:13.904041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:13.904053 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:13.904109 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:13.904375 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.904998 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.906667 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.906969 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.907024 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:13.908604 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:13.908612 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:13.908613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:13.908621 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:13.910193 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:13.910474 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:13.910477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:13.910478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:13.910537 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:13.910694 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:13.910695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:13.910695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:13.910859 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:13.910941 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:13.911025 (+ 84us) server_negotiation.cc:299] Negotiation successful
0924 07:18:13.911100 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260924 07:18:13.911737 592 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:18:13.912025 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.911901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:35343 (local address 127.0.0.1:60110)
0924 07:18:13.911934 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.912001 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260924 07:18:13.912153 542 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
I20260924 07:18:13.912525 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.912430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:35343 (local address 127.0.0.1:60112)
0924 07:18:13.912474 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.912503 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 07:18:13.912619 592 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.252:35343: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.252.keytab.
Entry for principal kudu/127.10.130.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.252.keytab.
I20260924 07:18:13.918129 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.252:35343
--webserver_interface=127.10.130.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.252.keytab
--principal=kudu/127.10.130.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:46869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:42769,127.10.130.253:46637,127.10.130.252:35343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:18:13.961834 592 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } attempt: 1
I20260924 07:18:13.962303 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.962172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:35343 (local address 127.0.0.1:60128)
0924 07:18:13.962229 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.962272 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 07:18:13.962620 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.962537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:35343 (local address 127.0.0.1:60144)
0924 07:18:13.962577 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.962603 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260924 07:18:13.962699 592 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.252:35343: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
I20260924 07:18:13.970213 517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } attempt: 3
Sep 24 07:18:13 dist-test-slave-7q7m krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234293, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for kudu/127.10.130.253@KRBTEST.COM
I20260924 07:18:13.976683 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.970511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34044 (local address 127.10.130.253:46637)
0924 07:18:13.970560 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:13.970563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:13.970602 (+ 39us) server_negotiation.cc:407] Connection header received
0924 07:18:13.970629 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:13.970630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:13.970640 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:13.970707 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:13.970977 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.971757 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.972446 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.972796 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.972866 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:13.974563 (+ 1697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:13.974564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:13.974565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:13.974570 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:13.975719 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:13.976092 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:13.976093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:13.976093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:13.976137 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:13.976309 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:13.976309 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:13.976310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:13.976398 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:13.976438 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:13.976577 (+ 139us) server_negotiation.cc:299] Negotiation successful
0924 07:18:13.976628 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 07:18:13.976752 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.970449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:46637 (local address 127.0.0.1:34044)
0924 07:18:13.970497 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.970508 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:13.970551 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:13.970718 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:13.970719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:13.970853 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:13.970917 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:13.970919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.971827 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.971828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:13.972374 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:13.972376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:13.972861 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:13.972862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:13.972910 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:13.973330 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:13.973333 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:13.974487 (+ 1154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:13.975791 (+ 1304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:13.975792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:13.975793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:13.976026 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:13.976188 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:13.976189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:13.976189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:13.976257 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:13.976444 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:13.976445 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:13.976493 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 07:18:13.976562 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 07:18:13.976648 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260924 07:18:13.977246 517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:18:13.977551 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.977453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:35343 (local address 127.0.0.1:60156)
0924 07:18:13.977497 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.977527 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 07:18:13.977841 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:13.977761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:35343 (local address 127.0.0.1:60164)
0924 07:18:13.977797 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:13.977824 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 07:18:13.977926 517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.252:35343: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
W20260924 07:18:13.996517 599 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 07:18:13.996716 599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:13.996730 599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:13.996754 599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:13.997887 599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:13.997924 599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:13.997937 599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:13.997949 599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:13.997973 599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:13.999380 599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:46869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:42769,127.10.130.253:46637,127.10.130.252:35343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.252.keytab
--principal=kudu/127.10.130.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.252:35343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.130.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:13.999594 599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:13.999794 599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:14.004179 605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:14.004216 604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:14.004194 607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:14.004211 599 server_base.cc:1034] running on GCE node
I20260924 07:18:14.004500 599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:14.004761 599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:14.005888 599 hybrid_clock.cc:648] HybridClock initialized: now 1790234294005876 us; error 26 us; skew 500 ppm
Sep 24 07:18:14 dist-test-slave-7q7m krb5kdc[436](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234294, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:14.008136 599 init.cc:377] Logged in from keytab as kudu/127.10.130.252@KRBTEST.COM (short username kudu)
I20260924 07:18:14.008558 599 webserver.cc:460] Webserver started at http://127.10.130.252:43885/ using document root <none> and password file <none>
I20260924 07:18:14.008746 599 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:14.008790 599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:14.008909 599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:14.009057 517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } attempt: 1
I20260924 07:18:14.009398 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.009266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:35343 (local address 127.0.0.1:60176)
0924 07:18:14.009325 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:14.009362 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 07:18:14.009707 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.009616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:35343 (local address 127.0.0.1:60190)
0924 07:18:14.009653 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:14.009686 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 07:18:14.009788 517 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.252:35343: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
I20260924 07:18:14.010072 599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/data/instance:
uuid: "091c55e8ee9d41f38ac8175110ed7720"
format_stamp: "Formatted at 2026-09-24 07:18:14 on dist-test-slave-7q7m"
server_key: "5fedced40baec11c988ac1179922196e"
server_key_iv: "f0861d3502027455a06a336e37aad31c"
server_key_version: "encryptionkey@0"
I20260924 07:18:14.010353 599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/wal/instance:
uuid: "091c55e8ee9d41f38ac8175110ed7720"
format_stamp: "Formatted at 2026-09-24 07:18:14 on dist-test-slave-7q7m"
server_key: "5fedced40baec11c988ac1179922196e"
server_key_iv: "f0861d3502027455a06a336e37aad31c"
server_key_version: "encryptionkey@0"
I20260924 07:18:14.011448 599 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 07:18:14.012058 614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:14.012216 599 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:14.012288 599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/wal
uuid: "091c55e8ee9d41f38ac8175110ed7720"
format_stamp: "Formatted at 2026-09-24 07:18:14 on dist-test-slave-7q7m"
server_key: "5fedced40baec11c988ac1179922196e"
server_key_iv: "f0861d3502027455a06a336e37aad31c"
server_key_version: "encryptionkey@0"
I20260924 07:18:14.012354 599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:14.030825 592 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } attempt: 2
I20260924 07:18:14.031169 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.031063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:35343 (local address 127.0.0.1:60194)
0924 07:18:14.031114 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:14.031144 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 07:18:14.031422 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.031360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:35343 (local address 127.0.0.1:60198)
0924 07:18:14.031389 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:14.031408 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260924 07:18:14.031499 592 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.252:35343: Network error: Client connection negotiation failed: client connection to 127.10.130.252:35343: connect: Connection refused (error 111)
I20260924 07:18:14.039168 599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:14.041070 599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:14.041196 599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:14.044689 599 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.252:35343
I20260924 07:18:14.044728 666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.252:35343 every 8 connection(s)
I20260924 07:18:14.045089 599 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/data/info.pb
I20260924 07:18:14.045825 667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:14.047158 667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 07:18:14 dist-test-slave-7q7m krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234294, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.252@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:18:14.051930 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 599
I20260924 07:18:14.052045 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790234044967003-10763-0/minicluster-data/master-2/wal/instance
I20260924 07:18:14.052237 10763 external_mini_cluster.cc:1428] Setting key 75c7e4fe2184eb36b2a0eb3db3083344
I20260924 07:18:14.055001 519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.047711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55074 (local address 127.10.130.254:42769)
0924 07:18:14.047758 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.047759 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.047933 (+ 174us) server_negotiation.cc:407] Connection header received
0924 07:18:14.048414 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.048414 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.048422 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.048472 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.048985 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.049444 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.050286 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.050461 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.050503 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.052397 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:14.052398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:14.052399 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:14.052407 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:14.053851 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.054191 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.054192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.054192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.054249 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.054430 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.054431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.054431 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.054553 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:14.054600 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.054886 (+ 286us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.054945 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:18:14.055210 669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.047672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:42769 (local address 127.0.0.1:55074)
0924 07:18:14.047827 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:14.047864 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:14.048336 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:14.048528 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:14.048532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:14.048755 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:14.048924 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:14.048928 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.049499 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.049500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:14.050228 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:14.050230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.050511 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.050512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:14.050598 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.050960 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:14.050968 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:14.052312 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:14.053920 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:14.053926 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:14.053927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:14.054128 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:14.054302 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:14.054302 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:14.054302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:14.054382 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:14.054617 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:14.054622 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:14.054764 (+ 142us) client_negotiation.cc:769] Sending connection context
0924 07:18:14.054870 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 07:18:14.055006 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260924 07:18:14.055696 667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 07:18:14 dist-test-slave-7q7m krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.252@KRBTEST.COM
Sep 24 07:18:14 dist-test-slave-7q7m krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234294, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.252@KRBTEST.COM for kudu/127.10.130.253@KRBTEST.COM
I20260924 07:18:14.060500 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.052578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60214 (local address 127.10.130.252:35343)
0924 07:18:14.052707 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.052711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.052756 (+ 45us) server_negotiation.cc:407] Connection header received
0924 07:18:14.052814 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.052817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.052828 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.052898 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.053512 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.054403 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.055355 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.055731 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.055789 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.058208 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:14.058215 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:14.058215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:14.058221 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:14.059406 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.059749 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.059751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.059751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.059808 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.059968 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.059969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.059969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.060123 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:14.060267 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.060342 (+ 75us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.060409 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260924 07:18:14.062192 669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.055894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:46637 (local address 127.0.0.1:34056)
0924 07:18:14.056014 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:14.056024 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:14.056068 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:14.056180 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:14.056181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:14.056310 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:14.056359 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:14.056361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.057195 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.057196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:14.057645 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:14.057646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.058088 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.058089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:14.058143 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.058430 (+ 287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:14.058434 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:14.059664 (+ 1230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:14.061291 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:14.061293 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:14.061293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:14.061469 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:14.061647 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:14.061648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:14.061648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:14.061711 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:14.061920 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:14.061921 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:14.061962 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 07:18:14.062041 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 07:18:14.062117 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260924 07:18:14.062193 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.055947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34056 (local address 127.10.130.253:46637)
0924 07:18:14.056022 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.056023 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.056058 (+ 35us) server_negotiation.cc:407] Connection header received
0924 07:18:14.056108 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.056109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.056119 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.056170 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.056424 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.057126 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.057722 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.058011 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.058080 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.059752 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:14.059753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:14.059754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:14.059760 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:14.061092 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.061527 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.061528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.061529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.061586 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.061776 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.061776 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.061776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.061867 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:14.061911 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.062047 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.062105 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 07:18:14.062492 667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 07:18:14 dist-test-slave-7q7m krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
Sep 24 07:18:14 dist-test-slave-7q7m krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234294, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.252@KRBTEST.COM for kudu/127.10.130.252@KRBTEST.COM
I20260924 07:18:14.068822 519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.061286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55076 (local address 127.10.130.254:42769)
0924 07:18:14.061547 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.061549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.061556 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:18:14.061573 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.061574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.061582 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.061633 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.062114 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.062737 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.063625 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.063823 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.063884 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.066435 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:14.066436 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:14.066436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:14.066443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:14.067807 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.068166 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.068167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.068167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.068222 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.068376 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.068377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.068377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.068473 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:14.068643 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.068678 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.068743 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241}
I20260924 07:18:14.069037 669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.062684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:35343 (local address 127.0.0.1:60230)
0924 07:18:14.062765 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:14.062774 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:14.062814 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:14.062921 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:14.062921 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:14.063014 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:14.063038 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:14.063039 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.063785 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.063785 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:14.064417 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:14.064420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.064905 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.064906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:14.064937 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.065667 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:14.065671 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:14.066617 (+ 946us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:14.068168 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:14.068169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:14.068169 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:14.068326 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:14.068505 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:14.068505 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:14.068506 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:14.068561 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:14.068766 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:14.068766 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:14.068802 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 07:18:14.068863 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 07:18:14.068953 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260924 07:18:14.069468 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.062732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60230 (local address 127.10.130.252:35343)
0924 07:18:14.062762 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.062764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.062803 (+ 39us) server_negotiation.cc:407] Connection header received
0924 07:18:14.062853 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.062854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.062863 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.062918 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.063090 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.063729 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.064505 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.064816 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.064911 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.066694 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:14.066696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:14.066696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:14.066702 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:14.068099 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.068395 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.068395 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.068396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.068446 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.068613 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.068614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.068614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.068712 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:14.068767 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.068900 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.069399 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":436,"server-negotiator.queue_time_us":15}
I20260924 07:18:14.070668 667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720: Bootstrap starting.
I20260924 07:18:14.071465 667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:14.071746 667 log.cc:824] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:14.072500 667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720: No bootstrap required, opened a new log
Sep 24 07:18:14 dist-test-slave-7q7m krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.253@KRBTEST.COM
I20260924 07:18:14.073975 667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } }
I20260924 07:18:14.074102 667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 091c55e8ee9d41f38ac8175110ed7720, State: Initialized, Role: FOLLOWER
I20260924 07:18:14.074179 667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } }
I20260924 07:18:14.074447 667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:14.074427 673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } } }
I20260924 07:18:14.074522 673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 07:18:14.076308 684 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:18:14.076368 684 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:18:14.077005 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.069696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34064 (local address 127.10.130.253:46637)
0924 07:18:14.069779 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.069780 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.069787 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:18:14.069804 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.069805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.069812 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.069859 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.070287 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.070901 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.071529 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.072220 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.072291 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.074451 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:14.074453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:14.074453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:14.074459 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:14.075999 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.076348 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.076349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.076350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.076405 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.076590 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.076590 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.076590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.076690 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:14.076830 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.076866 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.076925 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 07:18:14.078158 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:18:14.076494 599 master_runner.cc:186] Error getting master registration for 127.10.130.254:42769: OK, instance_id { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" instance_seqno: 1790234293745449 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 07:18:14.078720 599 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 07:18:14.078943 491 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "10c09e06fcd14782bca4314762d8535b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55076
I20260924 07:18:14.084904 517 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } attempt: 2
Sep 24 07:18:14 dist-test-slave-7q7m krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234293, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for kudu/127.10.130.252@KRBTEST.COM
I20260924 07:18:14.092398 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.085254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60232 (local address 127.10.130.252:35343)
0924 07:18:14.085312 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.085315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.085324 (+ 9us) server_negotiation.cc:407] Connection header received
0924 07:18:14.085347 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.085347 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.085358 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.085421 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.085711 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.086409 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.087348 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.087633 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.087705 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.089754 (+ 2049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:14.089756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:14.089756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:14.089762 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:14.091225 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.091603 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.091604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.091604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.091654 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.091842 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.091843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.091843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.092015 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:14.092078 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.092232 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.092305 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:18:14.092406 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.085162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:35343 (local address 127.0.0.1:60232)
0924 07:18:14.085217 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:14.085227 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:14.085263 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:14.085439 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:14.085440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:14.085600 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:14.085644 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:14.085645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.086485 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.086486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:14.087275 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:14.087277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.087701 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.087702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:14.087751 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.088221 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:14.088225 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:14.089657 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:14.091304 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:14.091305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:14.091305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:14.091528 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:14.091716 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:14.091717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:14.091717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:14.091783 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:14.092075 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:14.092076 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:14.092124 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 07:18:14.092210 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 07:18:14.092309 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 07:18:14.093868 517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b: Bootstrap starting.
I20260924 07:18:14.094436 517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:14.094688 517 log.cc:824] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:14.095319 517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b: No bootstrap required, opened a new log
I20260924 07:18:14.096333 517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } }
I20260924 07:18:14.096437 517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c09e06fcd14782bca4314762d8535b, State: Initialized, Role: FOLLOWER
I20260924 07:18:14.096506 517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } }
I20260924 07:18:14.096720 687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } } }
I20260924 07:18:14.096782 517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:14.096799 687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:18:14.096901 491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 07:18:14.097023 491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:14.097822 491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } }
I20260924 07:18:14.098593 641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "10c09e06fcd14782bca4314762d8535b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "091c55e8ee9d41f38ac8175110ed7720"
I20260924 07:18:14.098728 641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720 [term 0 FOLLOWER]: Advancing to term 1
W20260924 07:18:14.098837 698 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:18:14.098881 698 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:18:14.098104 491 leader_election.cc:288] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [CANDIDATE]: Term 1 election: Requested vote from peers f064fed3299f4107997eba6c3d9b4c7e (127.10.130.253:46637), 091c55e8ee9d41f38ac8175110ed7720 (127.10.130.252:35343)
I20260924 07:18:14.099593 641 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10c09e06fcd14782bca4314762d8535b in term 1.
I20260924 07:18:14.100145 467 leader_election.cc:302] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 091c55e8ee9d41f38ac8175110ed7720, 10c09e06fcd14782bca4314762d8535b; no voters:
I20260924 07:18:14.100263 687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:18:14.100379 687 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [term 1 LEADER]: Becoming Leader. State: Replica: 10c09e06fcd14782bca4314762d8535b, State: Running, Role: LEADER
I20260924 07:18:14.100453 687 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } }
I20260924 07:18:14.100854 687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10c09e06fcd14782bca4314762d8535b. Latest consensus state: current_term: 1 leader_uuid: "10c09e06fcd14782bca4314762d8535b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } } }
I20260924 07:18:14.100924 687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:14.100174 449 master_runner.cc:186] Error getting master registration for 127.10.130.253:46637: OK, instance_id { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" instance_seqno: 1790234293890575 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 07:18:14.102347 449 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 07:18:14.108131 706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:18:14.108330 706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:18:14.109846 641 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720 [term 1 FOLLOWER]: Refusing update from remote peer 10c09e06fcd14782bca4314762d8535b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:18:14.110173 704 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [LEADER]: Connected to new peer: Peer: permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:18:14.111415 673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10c09e06fcd14782bca4314762d8535b. Latest consensus state: current_term: 1 leader_uuid: "10c09e06fcd14782bca4314762d8535b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } } }
I20260924 07:18:14.111508 673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:18:14.112748 704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "10c09e06fcd14782bca4314762d8535b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } } }
I20260924 07:18:14.112854 704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:14.113209 673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "10c09e06fcd14782bca4314762d8535b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } } }
I20260924 07:18:14.113298 673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:18:14.113514 710 mvcc.cc:205] Tried to move back new op lower bound from 7332799668671680512 to 7332799668636102656. Current Snapshot: MvccSnapshot[applied={T|T < 7332799668671680512}]
I20260924 07:18:14.114032 711 mvcc.cc:205] Tried to move back new op lower bound from 7332799668671680512 to 7332799668636102656. Current Snapshot: MvccSnapshot[applied={T|T < 7332799668671680512}]
I20260924 07:18:14.114480 706 catalog_manager.cc:1347] Generated new cluster ID: 5c0d70d943024961b0f90d5ce0d07f30
I20260924 07:18:14.114542 706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:18:14.119031 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.108227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34080 (local address 127.10.130.253:46637)
0924 07:18:14.108272 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.108274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.108517 (+ 243us) server_negotiation.cc:407] Connection header received
0924 07:18:14.108664 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.108666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.108688 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.108731 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.110012 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.110759 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.112586 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.112905 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.112947 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.115638 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:14.115640 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:14.115640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:14.115660 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:14.117364 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.117910 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.117911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.117912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.117984 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.118277 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.118279 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.118280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.118427 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:14.118515 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.118864 (+ 349us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.118941 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:18:14.122627 592 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } attempt: 3
I20260924 07:18:14.124122 706 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:18:14.124779 706 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:18:14.126863 519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.108502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55092 (local address 127.10.130.254:42769)
0924 07:18:14.108536 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.108537 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.115036 (+ 6499us) server_negotiation.cc:407] Connection header received
0924 07:18:14.115131 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.115132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.115143 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.115183 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.115744 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.116463 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.117486 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.117756 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.117796 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.124131 (+ 6335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:14.124132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:14.124132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:14.124139 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:14.125658 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.126007 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.126008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.126009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.126064 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.126283 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.126284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.126284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.126406 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:14.126474 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.126679 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.126775 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:18:14.129320 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.108812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60234 (local address 127.10.130.252:35343)
0924 07:18:14.108848 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.108850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.119944 (+ 11094us) server_negotiation.cc:407] Connection header received
0924 07:18:14.120093 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.120094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.120104 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.120143 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.120958 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.121708 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.123160 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.123460 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.123497 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.125631 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:14.125632 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:14.125633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:14.125640 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:14.127307 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.128248 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.128249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.128250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.128318 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.128615 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.128616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.128617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.128763 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:14.128836 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.129144 (+ 308us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.129222 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 24 07:18:14 dist-test-slave-7q7m krb5kdc[436](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234293, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.253@KRBTEST.COM for kudu/127.10.130.252@KRBTEST.COM
I20260924 07:18:14.138571 706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b: Generated new TSK 0
I20260924 07:18:14.138787 706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 07:18:14.139384 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.122858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:35343 (local address 127.0.0.1:60248)
0924 07:18:14.122940 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:14.122950 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:14.122998 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:14.129632 (+ 6634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:14.129633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:14.129791 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:14.129836 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:14.129837 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.130834 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.130835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:14.131645 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:14.131648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.132212 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.132213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:14.132284 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.132745 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:14.132750 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:14.134258 (+ 1508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:14.136689 (+ 2431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:14.136690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:14.136690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:14.136903 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:14.137184 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:14.137185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:14.137185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:14.137255 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:14.137671 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:14.137672 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:14.139105 (+ 1433us) client_negotiation.cc:769] Sending connection context
0924 07:18:14.139180 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 07:18:14.139282 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260924 07:18:14.140086 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.123075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60248 (local address 127.10.130.252:35343)
0924 07:18:14.129392 (+ 6317us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.129394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.129401 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:18:14.129418 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.129419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.129427 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.129474 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.130017 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.130733 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.131756 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.132095 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.132149 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.134373 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:14.134374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:14.134374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:14.134396 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:14.136556 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.136996 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.136997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.136997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.137107 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:14.137332 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:14.137333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:14.137333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:14.137539 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:14.139639 (+ 2100us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.139718 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.139886 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6304,"thread_start_us":90,"threads_started":1}
I20260924 07:18:14.141975 592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e: Bootstrap starting.
I20260924 07:18:14.142729 592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:14.143044 592 log.cc:824] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:14.143898 592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e: No bootstrap required, opened a new log
I20260924 07:18:14.145890 592 raft_consensus.cc:348] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } }
I20260924 07:18:14.146028 592 raft_consensus.cc:729] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f064fed3299f4107997eba6c3d9b4c7e, State: Initialized, Role: FOLLOWER
I20260924 07:18:14.146108 592 consensus_queue.cc:260] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } }
I20260924 07:18:14.146433 592 sys_catalog.cc:564] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:14.146505 565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:14.147485 565 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e [term 1 FOLLOWER]: Refusing update from remote peer 10c09e06fcd14782bca4314762d8535b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:18:14.147657 566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "10c09e06fcd14782bca4314762d8535b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f064fed3299f4107997eba6c3d9b4c7e"
I20260924 07:18:14.148245 687 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:18:14.148730 719 sys_catalog.cc:455] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } } }
I20260924 07:18:14.148942 719 sys_catalog.cc:458] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:18:14.149012 566 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10c09e06fcd14782bca4314762d8535b in term 1.
I20260924 07:18:14.152722 730 catalog_manager.cc:1259] Loaded cluster ID: 5c0d70d943024961b0f90d5ce0d07f30
I20260924 07:18:14.152806 730 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e: loading cluster ID for follower catalog manager: success
I20260924 07:18:14.153995 730 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e: acquiring CA information for follower catalog manager: success
I20260924 07:18:14.154382 730 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 07:18:14.154734 719 sys_catalog.cc:455] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10c09e06fcd14782bca4314762d8535b. Latest consensus state: current_term: 1 leader_uuid: "10c09e06fcd14782bca4314762d8535b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } } }
I20260924 07:18:14.154848 719 sys_catalog.cc:458] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:18:14.154879 719 sys_catalog.cc:455] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "10c09e06fcd14782bca4314762d8535b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } } }
I20260924 07:18:14.154913 719 sys_catalog.cc:458] T 00000000000000000000000000000000 P f064fed3299f4107997eba6c3d9b4c7e [sys.catalog]: This master's current role is: FOLLOWER
W20260924 07:18:14.181895 519 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:14.180472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55102 (local address 127.10.130.254:42769)
0924 07:18:14.180508 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.180509 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.181272 (+ 763us) server_negotiation.cc:407] Connection header received
0924 07:18:14.181369 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.181370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.181383 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.181409 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.181829 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55102: BlockingRecv error: recv got EOF from 127.0.0.1:55102 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 07:18:14.182750 596 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:14.180254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34082 (local address 127.10.130.253:46637)
0924 07:18:14.180297 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.180300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.182096 (+ 1796us) server_negotiation.cc:407] Connection header received
0924 07:18:14.182258 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.182259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.182269 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.182296 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.182695 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34082: BlockingRecv error: recv got EOF from 127.0.0.1:34082 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 07:18:14.183365 671 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:14.180816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60262 (local address 127.10.130.252:35343)
0924 07:18:14.180850 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.180852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.182372 (+ 1520us) server_negotiation.cc:407] Connection header received
0924 07:18:14.182939 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.182940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.182949 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.182977 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.183292 (+ 315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60262: BlockingRecv error: recv got EOF from 127.0.0.1:60262 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:18:14.188309 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.184935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34096 (local address 127.10.130.253:46637)
0924 07:18:14.184966 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.184968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.185130 (+ 162us) server_negotiation.cc:407] Connection header received
0924 07:18:14.185239 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.185240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.185271 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.185309 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:14.185496 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.186219 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.187295 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.187593 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.187639 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.187867 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:14.188044 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:14.188099 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.188228 (+ 129us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.188246 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260924 07:18:14.191390 671 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:14.190660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60266 (local address 127.10.130.252:35343)
0924 07:18:14.190695 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.190697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.190908 (+ 211us) server_negotiation.cc:407] Connection header received
0924 07:18:14.191000 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.191001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.191015 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.191041 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.191345 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60266: BlockingRecv error: recv got EOF from 127.0.0.1:60266 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:18:14.196565 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.193473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34104 (local address 127.10.130.253:46637)
0924 07:18:14.193503 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.193504 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.193704 (+ 200us) server_negotiation.cc:407] Connection header received
0924 07:18:14.193790 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.193791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.193799 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.193823 (+ 24us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:14.193966 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.194668 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.195693 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.195969 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.196037 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.196245 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:14.196298 (+ 53us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:14.196326 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.196486 (+ 160us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.196504 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260924 07:18:14.199360 671 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:14.198671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60280 (local address 127.10.130.252:35343)
0924 07:18:14.198703 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.198704 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.198881 (+ 177us) server_negotiation.cc:407] Connection header received
0924 07:18:14.198967 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.198968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.198977 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.199005 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.199314 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60280: BlockingRecv error: recv got EOF from 127.0.0.1:60280 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:18:14.205416 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.202407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34118 (local address 127.10.130.253:46637)
0924 07:18:14.202437 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.202439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.202633 (+ 194us) server_negotiation.cc:407] Connection header received
0924 07:18:14.202722 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.202722 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.202731 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.202756 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:14.202908 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.203591 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.204669 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.204938 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.204981 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.205171 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:14.205213 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:14.205240 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.205342 (+ 102us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.205358 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260924 07:18:14.208101 671 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:14.207363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60296 (local address 127.10.130.252:35343)
0924 07:18:14.207395 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.207396 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.207613 (+ 217us) server_negotiation.cc:407] Connection header received
0924 07:18:14.207703 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.207704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.207713 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.207740 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.208059 (+ 319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60296: BlockingRecv error: recv got EOF from 127.0.0.1:60296 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
I20260924 07:18:14.216336 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.213115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34128 (local address 127.10.130.253:46637)
0924 07:18:14.213148 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.213150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.213369 (+ 219us) server_negotiation.cc:407] Connection header received
0924 07:18:14.213457 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.213458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.213466 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.213492 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:14.213652 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.214365 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.215496 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.215783 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.215825 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.216090 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:14.216121 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:14.216148 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.216257 (+ 109us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.216273 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 07:18:14.219046 671 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:14.218326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60298 (local address 127.10.130.252:35343)
0924 07:18:14.218357 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.218359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.218580 (+ 221us) server_negotiation.cc:407] Connection header received
0924 07:18:14.218667 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.218668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.218678 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.218705 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.219001 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60298: BlockingRecv error: recv got EOF from 127.0.0.1:60298 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:18:14.231259 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.228211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34142 (local address 127.10.130.253:46637)
0924 07:18:14.228245 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.228247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.228460 (+ 213us) server_negotiation.cc:407] Connection header received
0924 07:18:14.228549 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.228550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.228560 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.228585 (+ 25us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:14.228743 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.229474 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.230537 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.230811 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.230847 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.231021 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:14.231051 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:14.231076 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.231166 (+ 90us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.231193 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 07:18:14.233953 671 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:14.233232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60304 (local address 127.10.130.252:35343)
0924 07:18:14.233272 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.233274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.233468 (+ 194us) server_negotiation.cc:407] Connection header received
0924 07:18:14.233548 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.233549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.233560 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.233589 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.233906 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60304: BlockingRecv error: recv got EOF from 127.0.0.1:60304 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 07:18:14.254033 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.250975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.10.130.253:46637)
0924 07:18:14.251009 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.251010 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.251220 (+ 210us) server_negotiation.cc:407] Connection header received
0924 07:18:14.251308 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.251309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.251319 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.251343 (+ 24us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:14.251492 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.252257 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.253307 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.253596 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.253631 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.253802 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:14.253835 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:14.253858 (+ 23us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.253946 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.253963 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
W20260924 07:18:14.256925 671 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:14.256137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60316 (local address 127.10.130.252:35343)
0924 07:18:14.256183 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.256185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.256394 (+ 209us) server_negotiation.cc:407] Connection header received
0924 07:18:14.256478 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.256478 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.256490 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.256518 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.256865 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60316: BlockingRecv error: recv got EOF from 127.0.0.1:60316 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:18:14.292281 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.290098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34156 (local address 127.10.130.253:46637)
0924 07:18:14.290143 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.290145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.290233 (+ 88us) server_negotiation.cc:407] Connection header received
0924 07:18:14.290292 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.290293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.290303 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.290351 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:14.290469 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.290954 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.291673 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.291854 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.291906 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.292056 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:14.292088 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:14.292127 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.292186 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.292204 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260924 07:18:14.294356 671 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:14.293765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60332 (local address 127.10.130.252:35343)
0924 07:18:14.293820 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.293822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.293904 (+ 82us) server_negotiation.cc:407] Connection header received
0924 07:18:14.293958 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.293960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.293971 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.294010 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.294303 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60332: BlockingRecv error: recv got EOF from 127.0.0.1:60332 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 07:18:14.361747 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.359478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34160 (local address 127.10.130.253:46637)
0924 07:18:14.359543 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.359545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.359656 (+ 111us) server_negotiation.cc:407] Connection header received
0924 07:18:14.359728 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.359729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.359739 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.359804 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:14.359901 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.360407 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.361164 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.361356 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.361401 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.361528 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:14.361553 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:14.361585 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.361669 (+ 84us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.361684 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260924 07:18:14.363893 671 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:14.363360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60344 (local address 127.10.130.252:35343)
0924 07:18:14.363419 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.363421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.363442 (+ 21us) server_negotiation.cc:407] Connection header received
0924 07:18:14.363497 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.363498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.363509 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.363548 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.363839 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60344: BlockingRecv error: recv got EOF from 127.0.0.1:60344 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 07:18:14.495338 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.493003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34162 (local address 127.10.130.253:46637)
0924 07:18:14.493075 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.493077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.493152 (+ 75us) server_negotiation.cc:407] Connection header received
0924 07:18:14.493236 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.493237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.493249 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.493295 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:14.493414 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.493952 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.494718 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.494911 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.494964 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.495095 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:14.495131 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:14.495170 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.495238 (+ 68us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.495259 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260924 07:18:14.497359 671 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:14.496889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60352 (local address 127.10.130.252:35343)
0924 07:18:14.496942 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.496944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.496977 (+ 33us) server_negotiation.cc:407] Connection header received
0924 07:18:14.497027 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.497027 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.497037 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.497083 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.497318 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60352: BlockingRecv error: recv got EOF from 127.0.0.1:60352 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 07:18:14.571708 687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "10c09e06fcd14782bca4314762d8535b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c09e06fcd14782bca4314762d8535b" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 42769 } } peers { permanent_uuid: "f064fed3299f4107997eba6c3d9b4c7e" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 46637 } } peers { permanent_uuid: "091c55e8ee9d41f38ac8175110ed7720" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 35343 } } }
I20260924 07:18:14.571873 687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10c09e06fcd14782bca4314762d8535b [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:14.756460 596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:14.754417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.10.130.253:46637)
0924 07:18:14.754481 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.754482 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.754497 (+ 15us) server_negotiation.cc:407] Connection header received
0924 07:18:14.754587 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.754588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.754598 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.754652 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:14.754758 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.755235 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.755918 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:14.756122 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:14.756169 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:14.756284 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:14.756310 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:14.756341 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:14.756380 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 07:18:14.756393 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260924 07:18:14.758559 671 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:14.758078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60360 (local address 127.10.130.252:35343)
0924 07:18:14.758134 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:14.758136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:14.758158 (+ 22us) server_negotiation.cc:407] Connection header received
0924 07:18:14.758188 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:14.758189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:14.758198 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:14.758235 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:14.758499 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60360: BlockingRecv error: recv got EOF from 127.0.0.1:60360 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 07:18:15.076781 684 catalog_manager.cc:1259] Loaded cluster ID: 5c0d70d943024961b0f90d5ce0d07f30
I20260924 07:18:15.076843 684 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720: loading cluster ID for follower catalog manager: success
I20260924 07:18:15.077622 684 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720: acquiring CA information for follower catalog manager: success
I20260924 07:18:15.077900 684 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 091c55e8ee9d41f38ac8175110ed7720: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 07:18:15.273877 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:15.271612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.10.130.253:46637)
0924 07:18:15.271729 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:15.271731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:15.271757 (+ 26us) server_negotiation.cc:407] Connection header received
0924 07:18:15.271837 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:15.271837 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:15.271851 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:15.271901 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:15.272040 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:15.272530 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:15.273276 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:15.273468 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:15.273521 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:15.273639 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:15.273678 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:15.273718 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:15.273767 (+ 49us) server_negotiation.cc:299] Negotiation successful
0924 07:18:15.273790 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
I20260924 07:18:15.277733 860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:15.275412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60370 (local address 127.10.130.252:35343)
0924 07:18:15.275514 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:15.275516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:15.275523 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:18:15.275546 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:15.275547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:15.275594 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:15.275656 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:15.275774 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:15.276316 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:15.277010 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:15.277218 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:15.277270 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:15.277403 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:15.277513 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:15.277577 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:15.277623 (+ 46us) server_negotiation.cc:299] Negotiation successful
0924 07:18:15.277644 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":48,"threads_started":1}
I20260924 07:18:15.281965 867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:15.279120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55108 (local address 127.10.130.254:42769)
0924 07:18:15.279228 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:15.279230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:15.279317 (+ 87us) server_negotiation.cc:407] Connection header received
0924 07:18:15.279365 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:15.279366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:15.279441 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:15.279510 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:15.279595 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:15.280389 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:15.281147 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:15.281365 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:15.281419 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:15.281644 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:15.281740 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:15.281799 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:15.281863 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 07:18:15.281887 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":52,"threads_started":1}
I20260924 07:18:15.282087 860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:15.279366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60384 (local address 127.10.130.252:35343)
0924 07:18:15.279398 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:15.279399 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:15.279632 (+ 233us) server_negotiation.cc:407] Connection header received
0924 07:18:15.279707 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:15.279708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:15.279718 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:15.279744 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:15.279862 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:15.280361 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:15.281406 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:15.281659 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:15.281711 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:15.281867 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:15.281894 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:15.281919 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:15.282006 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 07:18:15.282027 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:18:15.282488 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:15.279143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34204 (local address 127.10.130.253:46637)
0924 07:18:15.279188 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:15.279189 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:15.279406 (+ 217us) server_negotiation.cc:407] Connection header received
0924 07:18:15.279474 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:15.279475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:15.279486 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:15.279533 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:15.279692 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:15.280419 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:15.281739 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:15.281992 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:15.282045 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:15.282260 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:15.282295 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:15.282320 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:15.282404 (+ 84us) server_negotiation.cc:299] Negotiation successful
0924 07:18:15.282426 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 07:18:15.283890 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 449
I20260924 07:18:15.288605 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 524
I20260924 07:18:15.293051 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 599
2026-09-24T07:18:15Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2750 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 07:18:15 dist-test-slave-7q7m krb5kdc[871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 07:18:15 dist-test-slave-7q7m krb5kdc[871](info): set up 2 sockets
Sep 24 07:18:15 dist-test-slave-7q7m krb5kdc[871](info): commencing operation
krb5kdc: starting...
W20260924 07:18:16.227841 10763 mini_kdc.cc:121] Time spent starting KDC: real 0.916s user 0.000s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/test-user.keytab.
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T07:18:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T07:18:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.130.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
Entry for principal kudu/127.10.130.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab.
I20260924 07:18:16.334375 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:46841
--webserver_interface=127.10.130.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:46841,127.10.130.253:35787,127.10.130.252:38915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 07:18:16.411715 884 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 07:18:16.411899 884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:16.411923 884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:16.411947 884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:16.413074 884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:16.413112 884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:16.413130 884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:16.413147 884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:16.413161 884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:16.414572 884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:46841,127.10.130.253:35787,127.10.130.252:38915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.254.keytab
--principal=kudu/127.10.130.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.254:46841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.130.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:16.414786 884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:16.414978 884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:16.419193 892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:16.419209 890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:16.419274 884 server_base.cc:1034] running on GCE node
W20260924 07:18:16.419257 889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:16.419581 884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:16.419862 884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:16.420950 884 hybrid_clock.cc:648] HybridClock initialized: now 1790234296420936 us; error 26 us; skew 500 ppm
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:16.423789 884 init.cc:377] Logged in from keytab as kudu/127.10.130.254@KRBTEST.COM (short username kudu)
I20260924 07:18:16.424293 884 webserver.cc:460] Webserver started at http://127.10.130.254:35373/ using document root <none> and password file <none>
I20260924 07:18:16.424527 884 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:16.424585 884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:16.424714 884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:16.425905 884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/data/instance:
uuid: "332711f5d8344517a12157baa1264b41"
format_stamp: "Formatted at 2026-09-24 07:18:16 on dist-test-slave-7q7m"
server_key: "9948d86c4bc157afe2be27294be9ba47"
server_key_iv: "7e4f09d7e2539ecb656ba0fa53447814"
server_key_version: "encryptionkey@0"
I20260924 07:18:16.426241 884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/wal/instance:
uuid: "332711f5d8344517a12157baa1264b41"
format_stamp: "Formatted at 2026-09-24 07:18:16 on dist-test-slave-7q7m"
server_key: "9948d86c4bc157afe2be27294be9ba47"
server_key_iv: "7e4f09d7e2539ecb656ba0fa53447814"
server_key_version: "encryptionkey@0"
I20260924 07:18:16.427651 884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260924 07:18:16.428479 899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:16.428622 884 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:16.428709 884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/wal
uuid: "332711f5d8344517a12157baa1264b41"
format_stamp: "Formatted at 2026-09-24 07:18:16 on dist-test-slave-7q7m"
server_key: "9948d86c4bc157afe2be27294be9ba47"
server_key_iv: "7e4f09d7e2539ecb656ba0fa53447814"
server_key_version: "encryptionkey@0"
I20260924 07:18:16.428773 884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:16.444199 884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:16.445726 884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:16.445842 884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:16.449353 884 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.254:46841
I20260924 07:18:16.449384 951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.254:46841 every 8 connection(s)
I20260924 07:18:16.449656 884 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/data/info.pb
I20260924 07:18:16.450374 952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:16.451709 952 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:18:16.458441 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 884
I20260924 07:18:16.458537 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-0/wal/instance
I20260924 07:18:16.458732 10763 external_mini_cluster.cc:1428] Setting key b362f24661eb7d85c8940d0361c3906d
I20260924 07:18:16.459939 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.452297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:46841 (local address 127.0.0.1:59742)
0924 07:18:16.452441 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.452478 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:16.452918 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:16.453161 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:16.453163 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:16.453394 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:16.453530 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.453535 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.454454 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.454455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.455090 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.455091 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.455531 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.455531 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.455594 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.456023 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:16.456036 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:16.457241 (+ 1205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:16.458669 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.458676 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.458678 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.458888 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.459115 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.459115 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.459116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.459197 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.459569 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:16.459572 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:16.459620 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 07:18:16.459685 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 07:18:16.459796 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260924 07:18:16.459964 955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.452322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59742 (local address 127.10.130.254:46841)
0924 07:18:16.452458 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.452468 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.452568 (+ 100us) server_negotiation.cc:407] Connection header received
0924 07:18:16.453057 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.453060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.453078 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.453155 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.453618 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.454374 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.455165 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.455465 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.455566 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.457315 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.457321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.457321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.457327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.458595 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.458963 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.458966 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.458966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.459057 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.459246 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.459246 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.459247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.459488 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.459589 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.459729 (+ 140us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.459867 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
I20260924 07:18:16.460621 952 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:18:16.460911 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.460800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:35787 (local address 127.0.0.1:46990)
0924 07:18:16.460840 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.460886 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:35787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260924 07:18:16.461040 902 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.130.253:35787: connect: Connection refused (error 111)
I20260924 07:18:16.461407 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.461319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:35787 (local address 127.0.0.1:47006)
0924 07:18:16.461359 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.461387 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:35787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 07:18:16.461498 952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.253:35787: Network error: Client connection negotiation failed: client connection to 127.10.130.253:35787: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.10.130.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.253.keytab.
Entry for principal kudu/127.10.130.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.253.keytab.
I20260924 07:18:16.478507 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.253:35787
--webserver_interface=127.10.130.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.253.keytab
--principal=kudu/127.10.130.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:46841,127.10.130.253:35787,127.10.130.252:38915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:18:16.510696 952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } attempt: 1
I20260924 07:18:16.511200 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.511032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:35787 (local address 127.0.0.1:47020)
0924 07:18:16.511095 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.511143 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:35787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260924 07:18:16.511555 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.511462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:35787 (local address 127.0.0.1:47036)
0924 07:18:16.511500 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.511530 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:35787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260924 07:18:16.511652 952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.253:35787: Network error: Client connection negotiation failed: client connection to 127.10.130.253:35787: connect: Connection refused (error 111)
W20260924 07:18:16.554000 959 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 07:18:16.554179 959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:16.554194 959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:16.554212 959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:16.555336 959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:16.555374 959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:16.555387 959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:16.555410 959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:16.555425 959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:16.556892 959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:46841,127.10.130.253:35787,127.10.130.252:38915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.253.keytab
--principal=kudu/127.10.130.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.253:35787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.130.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:16.557093 959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:16.557293 959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:16.561568 967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:16.561605 965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:16.561697 959 server_base.cc:1034] running on GCE node
W20260924 07:18:16.561574 964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:16.561930 959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:16.562201 959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:16.563336 959 hybrid_clock.cc:648] HybridClock initialized: now 1790234296563322 us; error 27 us; skew 500 ppm
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:16.565866 959 init.cc:377] Logged in from keytab as kudu/127.10.130.253@KRBTEST.COM (short username kudu)
I20260924 07:18:16.566236 959 webserver.cc:460] Webserver started at http://127.10.130.253:44945/ using document root <none> and password file <none>
I20260924 07:18:16.566428 959 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:16.566473 959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:16.566624 959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:16.567574 959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/data/instance:
uuid: "bff88741c4f54f058086e4710dfa1b26"
format_stamp: "Formatted at 2026-09-24 07:18:16 on dist-test-slave-7q7m"
server_key: "d63ce5bbb10388a3061c160d66a0157a"
server_key_iv: "fccde52dfab95d0a815e7ab3f977fdd5"
server_key_version: "encryptionkey@0"
I20260924 07:18:16.567874 959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/wal/instance:
uuid: "bff88741c4f54f058086e4710dfa1b26"
format_stamp: "Formatted at 2026-09-24 07:18:16 on dist-test-slave-7q7m"
server_key: "d63ce5bbb10388a3061c160d66a0157a"
server_key_iv: "fccde52dfab95d0a815e7ab3f977fdd5"
server_key_version: "encryptionkey@0"
I20260924 07:18:16.569025 959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 07:18:16.569633 974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:16.569773 959 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 07:18:16.569849 959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/wal
uuid: "bff88741c4f54f058086e4710dfa1b26"
format_stamp: "Formatted at 2026-09-24 07:18:16 on dist-test-slave-7q7m"
server_key: "d63ce5bbb10388a3061c160d66a0157a"
server_key_iv: "fccde52dfab95d0a815e7ab3f977fdd5"
server_key_version: "encryptionkey@0"
I20260924 07:18:16.569911 959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:16.579792 952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } attempt: 2
I20260924 07:18:16.580133 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.580023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:35787 (local address 127.0.0.1:47050)
0924 07:18:16.580071 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.580105 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:35787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260924 07:18:16.580453 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.580390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:35787 (local address 127.0.0.1:47058)
0924 07:18:16.580418 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.580439 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.253:35787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260924 07:18:16.580524 952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.253:35787: Network error: Client connection negotiation failed: client connection to 127.10.130.253:35787: connect: Connection refused (error 111)
I20260924 07:18:16.582422 959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:16.583925 959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:16.584071 959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:16.587507 959 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.253:35787
I20260924 07:18:16.587528 1026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.253:35787 every 8 connection(s)
I20260924 07:18:16.587808 959 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/data/info.pb
I20260924 07:18:16.588583 1027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:16.589922 1027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:18:16.591990 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 959
I20260924 07:18:16.592087 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-1/wal/instance
I20260924 07:18:16.592255 10763 external_mini_cluster.cc:1428] Setting key fc16cf919b29a2892c363c274c8a3f50
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.253@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:18:16.598284 955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.590519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59750 (local address 127.10.130.254:46841)
0924 07:18:16.590584 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.590586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.590720 (+ 134us) server_negotiation.cc:407] Connection header received
0924 07:18:16.591181 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.591182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.591191 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.591229 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.591745 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.592336 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.593263 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.593529 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.593586 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.595790 (+ 2204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.595791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.595791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.595798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.597013 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.597452 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.597453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.597453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.597498 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.597734 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.597734 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.597735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.597825 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.597868 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.598158 (+ 290us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.598224 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 07:18:16.598517 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.590478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:46841 (local address 127.0.0.1:59750)
0924 07:18:16.590616 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.590651 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:16.591088 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:16.591291 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:16.591294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:16.591511 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:16.591686 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.591690 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.592415 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.592416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.593169 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.593172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.593601 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.593602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.593747 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.594194 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:16.594202 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:16.595700 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:16.597091 (+ 1391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.597098 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.597100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.597379 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.597548 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.597549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.597549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.597677 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.597877 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:16.597882 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:16.598031 (+ 149us) client_negotiation.cc:769] Sending connection context
0924 07:18:16.598150 (+ 119us) client_negotiation.cc:240] Negotiation successful
0924 07:18:16.598294 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260924 07:18:16.599062 1027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.130.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.130.252@KRBTEST.COM" created.
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.253@KRBTEST.COM for kudu/127.10.130.253@KRBTEST.COM
I20260924 07:18:16.605422 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.599279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:35787 (local address 127.0.0.1:47074)
0924 07:18:16.599327 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.599339 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:16.599379 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:16.599555 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:16.599556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:16.599654 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:16.599697 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.599698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.600503 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.600504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.601034 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.601036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.601541 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.601542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.601597 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.601926 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:16.601929 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:16.602929 (+ 1000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:16.604593 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.604594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.604594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.604742 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.604930 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.604930 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.604931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.604972 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.605228 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:16.605229 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:16.605259 (+ 30us) client_negotiation.cc:769] Sending connection context
0924 07:18:16.605302 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 07:18:16.605363 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 07:18:16.605553 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.599315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47074 (local address 127.10.130.253:35787)
0924 07:18:16.599436 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.599440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.599456 (+ 16us) server_negotiation.cc:407] Connection header received
0924 07:18:16.599477 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.599478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.599489 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.599545 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.599747 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.600428 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.601105 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.601454 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.601524 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.602997 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.603005 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.603005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.603012 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.604522 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.604813 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.604816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.604817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.604873 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.605009 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.605009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.605010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.605173 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.605254 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.605336 (+ 82us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.605419 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":96,"thread_start_us":65,"threads_started":1}
I20260924 07:18:16.606056 1027 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:18:16.606325 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.606233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38915 (local address 127.0.0.1:55034)
0924 07:18:16.606270 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.606308 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260924 07:18:16.606455 975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38915: connect: Connection refused (error 111)
I20260924 07:18:16.606863 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.606786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38915 (local address 127.0.0.1:55036)
0924 07:18:16.606825 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.606847 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 07:18:16.606958 1027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.252:38915: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38915: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.130.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.252.keytab.
Entry for principal kudu/127.10.130.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.252.keytab.
I20260924 07:18:16.614454 10763 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9LiIRP/build/release/bin/kudu
/tmp/dist-test-task9LiIRP/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.252:38915
--webserver_interface=127.10.130.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.252.keytab
--principal=kudu/127.10.130.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.130.212:41039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.130.254:46841,127.10.130.253:35787,127.10.130.252:38915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 07:18:16.656155 1027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } attempt: 1
I20260924 07:18:16.656584 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.656473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38915 (local address 127.0.0.1:55042)
0924 07:18:16.656523 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.656554 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 07:18:16.656909 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.656828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38915 (local address 127.0.0.1:55054)
0924 07:18:16.656866 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.656889 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260924 07:18:16.656994 1027 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.252:38915: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38915: connect: Connection refused (error 111)
I20260924 07:18:16.671663 952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } attempt: 3
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for kudu/127.10.130.253@KRBTEST.COM
I20260924 07:18:16.678597 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.671937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:35787 (local address 127.0.0.1:47088)
0924 07:18:16.672011 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.672021 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:16.672065 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:16.672281 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:16.672282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:16.672414 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:16.672472 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.672474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.673399 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.673400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.673935 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.673937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.674380 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.674381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.674428 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.674823 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:16.674827 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:16.676202 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:16.677772 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.677772 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.677773 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.677921 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.678104 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.678104 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.678104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.678149 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.678396 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:16.678396 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:16.678433 (+ 37us) client_negotiation.cc:769] Sending connection context
0924 07:18:16.678472 (+ 39us) client_negotiation.cc:240] Negotiation successful
0924 07:18:16.678536 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260924 07:18:16.678674 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.672043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.10.130.253:35787)
0924 07:18:16.672133 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.672135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.672147 (+ 12us) server_negotiation.cc:407] Connection header received
0924 07:18:16.672175 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.672175 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.672188 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.672275 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.672540 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.673324 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.674012 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.674322 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.674386 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.676281 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.676284 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.676284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.676293 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.677699 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.677985 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.677986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.677986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.678043 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.678197 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.678198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.678198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.678337 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.678404 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.678511 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.678587 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 07:18:16.679008 952 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:18:16.679314 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.679222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38915 (local address 127.0.0.1:55068)
0924 07:18:16.679262 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.679287 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260924 07:18:16.679592 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.679537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38915 (local address 127.0.0.1:55076)
0924 07:18:16.679561 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.679580 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260924 07:18:16.679668 952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.252:38915: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38915: connect: Connection refused (error 111)
W20260924 07:18:16.692343 1034 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 07:18:16.692515 1034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 07:18:16.692538 1034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 07:18:16.692557 1034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 07:18:16.693627 1034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 07:18:16.693660 1034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 07:18:16.693672 1034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 07:18:16.693683 1034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 07:18:16.693699 1034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 07:18:16.695042 1034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.130.212:41039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.130.254:46841,127.10.130.253:35787,127.10.130.252:38915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/krb5kdc/kudu_127.10.130.252.keytab
--principal=kudu/127.10.130.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.130.252:38915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.130.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 06:13:45 UTC on ccycloud.quasar-lsaogh.root.comops.site
build id 4669408
I20260924 07:18:16.695207 1034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 07:18:16.695362 1034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 07:18:16.699626 1040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:16.699671 1034 server_base.cc:1034] running on GCE node
W20260924 07:18:16.699568 1042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 07:18:16.699574 1039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 07:18:16.699958 1034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 07:18:16.700245 1034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 07:18:16.701375 1034 hybrid_clock.cc:648] HybridClock initialized: now 1790234296701361 us; error 24 us; skew 500 ppm
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 07:18:16.703449 1034 init.cc:377] Logged in from keytab as kudu/127.10.130.252@KRBTEST.COM (short username kudu)
I20260924 07:18:16.703814 1034 webserver.cc:460] Webserver started at http://127.10.130.252:39423/ using document root <none> and password file <none>
I20260924 07:18:16.704013 1034 fs_manager.cc:359] Metadata directory not provided
I20260924 07:18:16.704062 1034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 07:18:16.704180 1034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 07:18:16.705118 1034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/data/instance:
uuid: "3b83b4b6a2964651aa70de12c5db2a86"
format_stamp: "Formatted at 2026-09-24 07:18:16 on dist-test-slave-7q7m"
server_key: "4d0f745735b2a66f36100c051ee2c081"
server_key_iv: "e3475bc919464aa07476a17779d74506"
server_key_version: "encryptionkey@0"
I20260924 07:18:16.705430 1034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/wal/instance:
uuid: "3b83b4b6a2964651aa70de12c5db2a86"
format_stamp: "Formatted at 2026-09-24 07:18:16 on dist-test-slave-7q7m"
server_key: "4d0f745735b2a66f36100c051ee2c081"
server_key_iv: "e3475bc919464aa07476a17779d74506"
server_key_version: "encryptionkey@0"
I20260924 07:18:16.706550 1034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 07:18:16.707165 1049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 07:18:16.707326 1034 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 07:18:16.707403 1034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/data,/tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/wal
uuid: "3b83b4b6a2964651aa70de12c5db2a86"
format_stamp: "Formatted at 2026-09-24 07:18:16 on dist-test-slave-7q7m"
server_key: "4d0f745735b2a66f36100c051ee2c081"
server_key_iv: "e3475bc919464aa07476a17779d74506"
server_key_version: "encryptionkey@0"
I20260924 07:18:16.707451 1034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 07:18:16.710801 952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } attempt: 1
I20260924 07:18:16.711110 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.711020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38915 (local address 127.0.0.1:55080)
0924 07:18:16.711063 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.711090 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260924 07:18:16.711416 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.711329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38915 (local address 127.0.0.1:55094)
0924 07:18:16.711368 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.711392 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 07:18:16.711509 952 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.130.252:38915: Network error: Client connection negotiation failed: client connection to 127.10.130.252:38915: connect: Connection refused (error 111)
I20260924 07:18:16.718621 1034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 07:18:16.720475 1034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 07:18:16.720580 1034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 07:18:16.723969 1034 rpc_server.cc:307] RPC server started. Bound to: 127.10.130.252:38915
I20260924 07:18:16.724051 1101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.130.252:38915 every 8 connection(s)
I20260924 07:18:16.724337 1034 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/data/info.pb
I20260924 07:18:16.725085 1102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 07:18:16.725123 1027 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } attempt: 2
I20260924 07:18:16.727083 1102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:18:16.728188 10763 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9LiIRP/build/release/bin/kudu as pid 1034
I20260924 07:18:16.728284 10763 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9LiIRP/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790234044967003-10763-0/minicluster-data/master-2/wal/instance
I20260924 07:18:16.728469 10763 external_mini_cluster.cc:1428] Setting key 67255e7d1f988c451c3a262f34c8eaab
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.253@KRBTEST.COM for kudu/127.10.130.252@KRBTEST.COM
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.252@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.252@KRBTEST.COM
I20260924 07:18:16.734592 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.725374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38915 (local address 127.0.0.1:55096)
0924 07:18:16.725427 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.725438 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:16.725486 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:16.726382 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:16.726384 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:16.726542 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:16.726584 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.726586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.727762 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.727763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.728494 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.728507 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.729106 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.729107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.729162 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.729518 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:16.729523 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:16.730807 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:16.733338 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.733340 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.733340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.733567 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.733823 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.733824 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.733825 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.733889 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.734285 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:16.734286 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:16.734331 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 07:18:16.734394 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 07:18:16.734480 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 07:18:16.734731 955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.727638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.10.130.254:46841)
0924 07:18:16.727688 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.727691 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.727820 (+ 129us) server_negotiation.cc:407] Connection header received
0924 07:18:16.727884 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.727885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.727894 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.727935 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.728376 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.728998 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.729826 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.730082 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.730135 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.732295 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.732296 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.732296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.732303 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.733672 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.733982 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.733984 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.733984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.734033 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.734213 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.734214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.734214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.734331 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.734384 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.734573 (+ 189us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.734643 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 07:18:16.735059 1104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.725492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55096 (local address 127.10.130.252:38915)
0924 07:18:16.725620 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.725628 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.725649 (+ 21us) server_negotiation.cc:407] Connection header received
0924 07:18:16.726219 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.726225 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.726269 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.726384 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.726672 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.727660 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.728596 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.729038 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.729130 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.730895 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.730900 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.730900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.730906 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.733261 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.733644 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.733647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.733647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.733733 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.733958 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.733959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.733959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.734229 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.734301 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.734717 (+ 416us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.734837 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":62,"threads_started":1}
I20260924 07:18:16.735150 1105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.727586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.254:46841 (local address 127.0.0.1:59754)
0924 07:18:16.727727 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.727759 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:16.727822 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:16.727950 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:16.727953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:16.728203 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:16.728271 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.728273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.729087 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.729088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.729751 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.729753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.730147 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.730148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.730220 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.730634 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:16.730641 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:16.732226 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:16.733730 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.733736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.733739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.733925 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.734092 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.734093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.734093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.734154 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.734376 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:16.734379 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:16.734437 (+ 58us) client_negotiation.cc:769] Sending connection context
0924 07:18:16.734911 (+ 474us) client_negotiation.cc:240] Negotiation successful
0924 07:18:16.735018 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
I20260924 07:18:16.736066 1102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } has no permanent_uuid. Determining permanent_uuid...
I20260924 07:18:16.737192 1027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26: Bootstrap starting.
I20260924 07:18:16.738135 1107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.729103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55098 (local address 127.10.130.252:38915)
0924 07:18:16.729853 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.729855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.729860 (+ 5us) server_negotiation.cc:407] Connection header received
0924 07:18:16.729875 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.729875 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.729883 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.729938 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.730600 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.731236 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.732246 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.732530 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.732588 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.734922 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.734923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.734923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.734929 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.736969 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.737332 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.737333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.737333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.737389 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.737596 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.737597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.737598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.737730 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.737865 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.737934 (+ 69us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.738012 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":80,"threads_started":1}
I20260924 07:18:16.738178 1027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:16.738467 1027 log.cc:824] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:16.739770 1027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26: No bootstrap required, opened a new log
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.252@KRBTEST.COM for kudu/127.10.130.253@KRBTEST.COM
I20260924 07:18:16.741528 1027 raft_consensus.cc:348] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } }
I20260924 07:18:16.741667 1027 raft_consensus.cc:729] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff88741c4f54f058086e4710dfa1b26, State: Initialized, Role: FOLLOWER
I20260924 07:18:16.741742 1027 consensus_queue.cc:260] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } }
I20260924 07:18:16.742027 1027 sys_catalog.cc:564] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:16.742666 1105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.736287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.253:35787 (local address 127.0.0.1:47102)
0924 07:18:16.736325 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.736334 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:16.736364 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:16.736479 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:16.736480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:16.736629 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:16.736660 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.736661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.737374 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.737375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.738004 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.738006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.738597 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.738598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.738655 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.739001 (+ 346us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:16.739005 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:16.740251 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:16.741726 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.741727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.741727 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.741917 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.742115 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.742116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.742117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.742169 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.742401 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:16.742402 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:16.742441 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 07:18:16.742498 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 07:18:16.742578 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260924 07:18:16.743072 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.736336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47102 (local address 127.10.130.253:35787)
0924 07:18:16.736371 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.736373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.736401 (+ 28us) server_negotiation.cc:407] Connection header received
0924 07:18:16.736420 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.736420 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.736429 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.736477 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.736727 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.737311 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.738185 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.738470 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.738536 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.740319 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.740320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.740320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.740326 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.741650 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.741984 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.741985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.741986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.742040 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.742223 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.742224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.742224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.742345 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.742399 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.742918 (+ 519us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.742987 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.254@KRBTEST.COM
I20260924 07:18:16.743613 1102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } has no permanent_uuid. Determining permanent_uuid...
W20260924 07:18:16.743911 1120 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:18:16.744021 1120 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:18:16.746814 955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.738644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59758 (local address 127.10.130.254:46841)
0924 07:18:16.738676 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.738677 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.738755 (+ 78us) server_negotiation.cc:407] Connection header received
0924 07:18:16.738809 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.738810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.738820 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.738872 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.739372 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.740060 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.741126 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.741417 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.741486 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.743960 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.743962 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.743962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.743968 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.745485 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.745836 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.745837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.745837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.745891 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.746083 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.746084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.746085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.746200 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.746328 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.746664 (+ 336us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.746729 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 07:18:16.744462 959 master_runner.cc:186] Error getting master registration for 127.10.130.254:46841: OK, instance_id { permanent_uuid: "332711f5d8344517a12157baa1264b41" instance_seqno: 1790234296448389 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 07:18:16.747839 959 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.252@KRBTEST.COM for kudu/127.10.130.252@KRBTEST.COM
I20260924 07:18:16.751072 1105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.743813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38915 (local address 127.0.0.1:55110)
0924 07:18:16.744114 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.744124 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:16.744160 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:16.744479 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:16.744479 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:16.744605 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:16.744632 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.744633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.745435 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.745436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.746143 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.746145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.746582 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.746583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.746625 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.746970 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:16.746976 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:16.748268 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:16.750140 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.750141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.750142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.750313 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.750505 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.750506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.750506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.750555 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.750784 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:16.750784 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:16.750819 (+ 35us) client_negotiation.cc:769] Sending connection context
0924 07:18:16.750878 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 07:18:16.750965 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287}
I20260924 07:18:16.751119 1107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.744217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55110 (local address 127.10.130.252:38915)
0924 07:18:16.744273 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.744275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.744282 (+ 7us) server_negotiation.cc:407] Connection header received
0924 07:18:16.744298 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.744299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.744308 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.744354 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.744705 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.745357 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.746220 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.746494 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.746541 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.748359 (+ 1818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.748360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.748360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.748366 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.750069 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.750384 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.750385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.750385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.750440 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.750614 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.750615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.750615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.750730 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.750783 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.750901 (+ 118us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.751013 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":37}
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.130.253@KRBTEST.COM
I20260924 07:18:16.752054 1109 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } } }
I20260924 07:18:16.752136 1109 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:18:16.752457 1102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86: Bootstrap starting.
I20260924 07:18:16.753046 1102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:16.753299 1102 log.cc:824] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:16.754694 1102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86: No bootstrap required, opened a new log
I20260924 07:18:16.755221 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.747640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47116 (local address 127.10.130.253:35787)
0924 07:18:16.747672 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.747673 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.747735 (+ 62us) server_negotiation.cc:407] Connection header received
0924 07:18:16.747816 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.747817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.747825 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.747849 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.748385 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.749045 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.749667 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.749946 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.750007 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.752578 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.752579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.752579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.752585 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.754052 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.754354 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.754356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.754356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.754412 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.754618 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.754619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.754619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.754740 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.754871 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.755058 (+ 187us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.755138 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 07:18:16.756399 10763 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 07:18:16.756580 1102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } }
I20260924 07:18:16.756675 1102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b83b4b6a2964651aa70de12c5db2a86, State: Initialized, Role: FOLLOWER
I20260924 07:18:16.756722 1102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } }
I20260924 07:18:16.756913 1102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:16.757232 1123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } } }
I20260924 07:18:16.757360 1123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:18:16.757441 926 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "332711f5d8344517a12157baa1264b41"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59758
W20260924 07:18:16.758761 1134 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 07:18:16.758862 1134 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 07:18:16.758869 1034 master_runner.cc:186] Error getting master registration for 127.10.130.254:46841: OK, instance_id { permanent_uuid: "332711f5d8344517a12157baa1264b41" instance_seqno: 1790234296448389 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 07:18:16.760895 1034 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 07:18:16.786638 952 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } attempt: 2
Sep 24 07:18:16 dist-test-slave-7q7m krb5kdc[871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790234296, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.130.254@KRBTEST.COM for kudu/127.10.130.252@KRBTEST.COM
I20260924 07:18:16.793896 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.786966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.130.252:38915 (local address 127.0.0.1:55116)
0924 07:18:16.787027 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 07:18:16.787039 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 07:18:16.787080 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 07:18:16.787274 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 07:18:16.787275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 07:18:16.787465 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0924 07:18:16.787515 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.787517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.788199 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.788200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.789046 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 07:18:16.789048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.789505 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.789506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 07:18:16.789559 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.789997 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 07:18:16.790002 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 07:18:16.791300 (+ 1298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 07:18:16.792973 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.792974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.792975 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.793138 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.793297 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 07:18:16.793298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 07:18:16.793298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 07:18:16.793367 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 07:18:16.793601 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 07:18:16.793603 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 07:18:16.793646 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 07:18:16.793706 (+ 60us) client_negotiation.cc:240] Negotiation successful
0924 07:18:16.793792 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 07:18:16.794178 1107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.787046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55116 (local address 127.10.130.252:38915)
0924 07:18:16.787175 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.787176 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.787182 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:18:16.787202 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.787203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.787211 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.787252 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.787584 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.788118 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.789136 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.789436 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.789493 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.791388 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.791389 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.791390 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.791397 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.792907 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.793199 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.793201 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.793201 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.793253 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.793412 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.793413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.793413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.793543 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.793599 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.794021 (+ 422us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.794091 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260924 07:18:16.795780 952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41: Bootstrap starting.
I20260924 07:18:16.796494 952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41: Neither blocks nor log segments found. Creating new log.
I20260924 07:18:16.796801 952 log.cc:824] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41: Log is configured to *not* fsync() on all Append() calls
I20260924 07:18:16.797588 952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41: No bootstrap required, opened a new log
I20260924 07:18:16.798875 952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } }
I20260924 07:18:16.799009 952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 332711f5d8344517a12157baa1264b41, State: Initialized, Role: FOLLOWER
I20260924 07:18:16.799093 952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } }
I20260924 07:18:16.799405 952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 07:18:16.799453 926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 07:18:16.799494 926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:16.800318 926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } }
I20260924 07:18:16.800565 926 leader_election.cc:288] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [CANDIDATE]: Term 1 election: Requested vote from peers bff88741c4f54f058086e4710dfa1b26 (127.10.130.253:35787), 3b83b4b6a2964651aa70de12c5db2a86 (127.10.130.252:38915)
I20260924 07:18:16.799377 1137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } } }
I20260924 07:18:16.801096 1076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "332711f5d8344517a12157baa1264b41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3b83b4b6a2964651aa70de12c5db2a86"
I20260924 07:18:16.801080 1001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "332711f5d8344517a12157baa1264b41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bff88741c4f54f058086e4710dfa1b26"
I20260924 07:18:16.801227 1076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:16.801275 1001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26 [term 0 FOLLOWER]: Advancing to term 1
I20260924 07:18:16.801194 1137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:18:16.802094 1076 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 332711f5d8344517a12157baa1264b41 in term 1.
I20260924 07:18:16.802206 1001 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 332711f5d8344517a12157baa1264b41 in term 1.
I20260924 07:18:16.802302 900 leader_election.cc:302] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 332711f5d8344517a12157baa1264b41, 3b83b4b6a2964651aa70de12c5db2a86; no voters:
I20260924 07:18:16.802449 1137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 07:18:16.802587 1137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [term 1 LEADER]: Becoming Leader. State: Replica: 332711f5d8344517a12157baa1264b41, State: Running, Role: LEADER
I20260924 07:18:16.802671 1137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } }
I20260924 07:18:16.803725 1154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 332711f5d8344517a12157baa1264b41. Latest consensus state: current_term: 1 leader_uuid: "332711f5d8344517a12157baa1264b41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } } }
I20260924 07:18:16.803807 1154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:16.804173 1156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 07:18:16.804678 1156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 07:18:16.806049 1076 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86 [term 1 FOLLOWER]: Refusing update from remote peer 332711f5d8344517a12157baa1264b41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:18:16.806384 1154 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:18:16.807636 1123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 332711f5d8344517a12157baa1264b41. Latest consensus state: current_term: 1 leader_uuid: "332711f5d8344517a12157baa1264b41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } } }
I20260924 07:18:16.807716 1123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:18:16.808110 1001 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26 [term 1 FOLLOWER]: Refusing update from remote peer 332711f5d8344517a12157baa1264b41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 07:18:16.808389 1154 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 07:18:16.809566 1109 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 332711f5d8344517a12157baa1264b41. Latest consensus state: current_term: 1 leader_uuid: "332711f5d8344517a12157baa1264b41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } } }
I20260924 07:18:16.809649 1109 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:18:16.809755 955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.802416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59770 (local address 127.10.130.254:46841)
0924 07:18:16.802556 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.802558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.802564 (+ 6us) server_negotiation.cc:407] Connection header received
0924 07:18:16.802581 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.802581 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.802589 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.802633 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.803478 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.804181 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.805131 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.805386 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.805443 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.806837 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.806838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.806838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.806844 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.808381 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.808731 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.808732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.808733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.808786 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.808965 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.808966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.808966 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.809096 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.809178 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.809588 (+ 410us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.809660 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260924 07:18:16.810654 1154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "332711f5d8344517a12157baa1264b41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } } }
I20260924 07:18:16.810725 1154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:16.811131 1109 sys_catalog.cc:455] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "332711f5d8344517a12157baa1264b41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } } }
I20260924 07:18:16.811216 1109 sys_catalog.cc:458] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:18:16.811342 1158 mvcc.cc:205] Tried to move back new op lower bound from 7332799679715454976 to 7332799679704375296. Current Snapshot: MvccSnapshot[applied={T|T < 7332799679715454976}]
I20260924 07:18:16.812252 1163 mvcc.cc:205] Tried to move back new op lower bound from 7332799679715454976 to 7332799679704375296. Current Snapshot: MvccSnapshot[applied={T|T < 7332799679715454976}]
I20260924 07:18:16.812814 1137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "332711f5d8344517a12157baa1264b41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } } }
I20260924 07:18:16.812893 1137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41 [sys.catalog]: This master's current role is: LEADER
I20260924 07:18:16.813431 1159 mvcc.cc:205] Tried to move back new op lower bound from 7332799679715454976 to 7332799679704375296. Current Snapshot: MvccSnapshot[applied={T|T < 7332799679715454976}]
I20260924 07:18:16.813670 1156 catalog_manager.cc:1347] Generated new cluster ID: 350a6d1dd0fc4d1a920298fc3c24bd54
I20260924 07:18:16.813711 1156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 07:18:16.813722 1123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "332711f5d8344517a12157baa1264b41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332711f5d8344517a12157baa1264b41" member_type: VOTER last_known_addr { host: "127.10.130.254" port: 46841 } } peers { permanent_uuid: "bff88741c4f54f058086e4710dfa1b26" member_type: VOTER last_known_addr { host: "127.10.130.253" port: 35787 } } peers { permanent_uuid: "3b83b4b6a2964651aa70de12c5db2a86" member_type: VOTER last_known_addr { host: "127.10.130.252" port: 38915 } } }
I20260924 07:18:16.813784 1123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 07:18:16.815377 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.808153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47128 (local address 127.10.130.253:35787)
0924 07:18:16.808191 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.808193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.808726 (+ 533us) server_negotiation.cc:407] Connection header received
0924 07:18:16.808813 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.808814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.808825 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.808848 (+ 23us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.809428 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.810149 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.811159 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.811431 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.811469 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.812835 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.812836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.812837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.812843 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.814276 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.814625 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.814626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.814626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.814675 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.814875 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.814876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.814877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.814989 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.815049 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.815237 (+ 188us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.815304 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:18:16.817870 1107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:16.808472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55130 (local address 127.10.130.252:38915)
0924 07:18:16.808509 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.808511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.809534 (+ 1023us) server_negotiation.cc:407] Connection header received
0924 07:18:16.809960 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.809961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.809970 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.809996 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.810613 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.811282 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.813968 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:16.814256 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:16.814290 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:16.815416 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 07:18:16.815417 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 07:18:16.815417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 07:18:16.815422 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 07:18:16.816906 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.817231 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.817232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.817232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.817274 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 07:18:16.817421 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 07:18:16.817421 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 07:18:16.817421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 07:18:16.817519 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 07:18:16.817577 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:16.817735 (+ 158us) server_negotiation.cc:299] Negotiation successful
0924 07:18:16.817792 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 07:18:16.826705 1156 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 07:18:16.827396 1156 catalog_manager.cc:1504] Loading token signing keys...
I20260924 07:18:16.834182 1156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 332711f5d8344517a12157baa1264b41: Generated new TSK 0
I20260924 07:18:16.834378 1156 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 07:18:16.859390 1031 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:16.857853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47134 (local address 127.10.130.253:35787)
0924 07:18:16.857894 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.857896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.858781 (+ 885us) server_negotiation.cc:407] Connection header received
0924 07:18:16.858881 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.858882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.858894 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.858922 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.859319 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47134: BlockingRecv error: recv got EOF from 127.0.0.1:47134 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 07:18:16.860390 955 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:16.858235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59782 (local address 127.10.130.254:46841)
0924 07:18:16.858272 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.858274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.859873 (+ 1599us) server_negotiation.cc:407] Connection header received
0924 07:18:16.859960 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.859961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.859971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.860039 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.860319 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59782: BlockingRecv error: recv got EOF from 127.0.0.1:59782 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260924 07:18:16.861186 1107 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:16.858583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55138 (local address 127.10.130.252:38915)
0924 07:18:16.858624 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.858626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.860719 (+ 2093us) server_negotiation.cc:407] Connection header received
0924 07:18:16.860806 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.860806 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.860818 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.860844 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.861129 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55138: BlockingRecv error: recv got EOF from 127.0.0.1:55138 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260924 07:18:16.863343 1031 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:16.862689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47138 (local address 127.10.130.253:35787)
0924 07:18:16.862722 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.862724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.862891 (+ 167us) server_negotiation.cc:407] Connection header received
0924 07:18:16.862982 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.862983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.862998 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.863024 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.863299 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47138: BlockingRecv error: recv got EOF from 127.0.0.1:47138 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 07:18:16.866230 1031 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:16.865550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47154 (local address 127.10.130.253:35787)
0924 07:18:16.865579 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.865580 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.865807 (+ 227us) server_negotiation.cc:407] Connection header received
0924 07:18:16.865889 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.865890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.865898 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.865923 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.866186 (+ 263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47154: BlockingRecv error: recv got EOF from 127.0.0.1:47154 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260924 07:18:16.869969 1031 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:16.869297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47166 (local address 127.10.130.253:35787)
0924 07:18:16.869327 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.869329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.869551 (+ 222us) server_negotiation.cc:407] Connection header received
0924 07:18:16.869633 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.869633 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.869642 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.869667 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.869931 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47166: BlockingRecv error: recv got EOF from 127.0.0.1:47166 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260924 07:18:16.875666 1031 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:16.874967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47174 (local address 127.10.130.253:35787)
0924 07:18:16.875009 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.875010 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.875223 (+ 213us) server_negotiation.cc:407] Connection header received
0924 07:18:16.875307 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.875308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.875316 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.875342 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.875626 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47174: BlockingRecv error: recv got EOF from 127.0.0.1:47174 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260924 07:18:16.885455 1031 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:16.884784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47176 (local address 127.10.130.253:35787)
0924 07:18:16.884814 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.884816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.885025 (+ 209us) server_negotiation.cc:407] Connection header received
0924 07:18:16.885108 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.885109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.885118 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.885143 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.885414 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47176: BlockingRecv error: recv got EOF from 127.0.0.1:47176 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260924 07:18:16.903362 1031 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:16.902566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47182 (local address 127.10.130.253:35787)
0924 07:18:16.902600 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.902601 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.902861 (+ 260us) server_negotiation.cc:407] Connection header received
0924 07:18:16.902955 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.902956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.902967 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.902995 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.903316 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47182: BlockingRecv error: recv got EOF from 127.0.0.1:47182 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 07:18:16.937484 1031 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:16.936721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47188 (local address 127.10.130.253:35787)
0924 07:18:16.936760 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:16.936762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:16.936900 (+ 138us) server_negotiation.cc:407] Connection header received
0924 07:18:16.936961 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:16.936962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:16.936974 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:16.937056 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:16.937421 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47188: BlockingRecv error: recv got EOF from 127.0.0.1:47188 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 07:18:17.003085 1031 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:17.002424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47194 (local address 127.10.130.253:35787)
0924 07:18:17.002487 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:17.002490 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:17.002570 (+ 80us) server_negotiation.cc:407] Connection header received
0924 07:18:17.002636 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:17.002637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:17.002651 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:17.002697 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:17.003020 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47194: BlockingRecv error: recv got EOF from 127.0.0.1:47194 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260924 07:18:17.132854 1031 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:17.132130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47200 (local address 127.10.130.253:35787)
0924 07:18:17.132197 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:17.132199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:17.132317 (+ 118us) server_negotiation.cc:407] Connection header received
0924 07:18:17.132398 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:17.132399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:17.132413 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:17.132460 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:17.132792 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47200: BlockingRecv error: recv got EOF from 127.0.0.1:47200 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260924 07:18:17.390476 1031 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 07:18:17.389880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.10.130.253:35787)
0924 07:18:17.389944 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:17.389946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:17.389971 (+ 25us) server_negotiation.cc:407] Connection header received
0924 07:18:17.390029 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:17.390030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:17.390043 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:17.390084 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 07:18:17.390412 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47214: BlockingRecv error: recv got EOF from 127.0.0.1:47214 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 07:18:17.744442 1120 catalog_manager.cc:1259] Loaded cluster ID: 350a6d1dd0fc4d1a920298fc3c24bd54
I20260924 07:18:17.744508 1120 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26: loading cluster ID for follower catalog manager: success
I20260924 07:18:17.745296 1120 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26: acquiring CA information for follower catalog manager: success
I20260924 07:18:17.745573 1120 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bff88741c4f54f058086e4710dfa1b26: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 07:18:17.759255 1134 catalog_manager.cc:1259] Loaded cluster ID: 350a6d1dd0fc4d1a920298fc3c24bd54
I20260924 07:18:17.759310 1134 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86: loading cluster ID for follower catalog manager: success
I20260924 07:18:17.760110 1134 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86: acquiring CA information for follower catalog manager: success
I20260924 07:18:17.760370 1134 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b83b4b6a2964651aa70de12c5db2a86: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 07:18:17.905947 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:17.903475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35400 (local address 127.10.130.253:35787)
0924 07:18:17.903581 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:17.903583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:17.903653 (+ 70us) server_negotiation.cc:407] Connection header received
0924 07:18:17.903721 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:17.903721 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:17.903777 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:17.903843 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:17.904017 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:17.904495 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:17.905229 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:17.905423 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:17.905476 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:17.905627 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:17.905736 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:17.905801 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:17.905838 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 07:18:17.905859 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":49,"threads_started":1}
I20260924 07:18:17.909592 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:17.907407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46620 (local address 127.10.130.252:38915)
0924 07:18:17.907495 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:17.907497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:17.907517 (+ 20us) server_negotiation.cc:407] Connection header received
0924 07:18:17.907549 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:17.907550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:17.907582 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:17.907644 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:17.907719 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:17.908241 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:17.908880 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:17.909079 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:17.909131 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:17.909224 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:17.909311 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:17.909374 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:17.909443 (+ 69us) server_negotiation.cc:299] Negotiation successful
0924 07:18:17.909463 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":45,"threads_started":1}
I20260924 07:18:17.913293 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:17.911009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46626 (local address 127.10.130.252:38915)
0924 07:18:17.911043 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:17.911045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:17.911253 (+ 208us) server_negotiation.cc:407] Connection header received
0924 07:18:17.911327 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:17.911328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:17.911336 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:17.911380 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:17.911494 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:17.911949 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:17.912680 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:17.912872 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:17.912926 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:17.913080 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:17.913110 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:17.913156 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:17.913207 (+ 51us) server_negotiation.cc:299] Negotiation successful
0924 07:18:17.913228 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 07:18:17.913625 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:17.910967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35412 (local address 127.10.130.253:35787)
0924 07:18:17.911009 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:17.911011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:17.911239 (+ 228us) server_negotiation.cc:407] Connection header received
0924 07:18:17.911278 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:17.911279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:17.911287 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:17.911326 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:17.911439 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:17.911929 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:17.912945 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:17.913199 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:17.913249 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:17.913407 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:17.913438 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:17.913479 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:17.913536 (+ 57us) server_negotiation.cc:299] Negotiation successful
0924 07:18:17.913556 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 07:18:17.914521 1245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 07:18:17.911020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41056 (local address 127.10.130.254:46841)
0924 07:18:17.911129 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0924 07:18:17.911131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 07:18:17.911139 (+ 8us) server_negotiation.cc:407] Connection header received
0924 07:18:17.911235 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 07:18:17.911236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 07:18:17.911288 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 07:18:17.911358 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 07:18:17.912067 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:17.912821 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:17.913735 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 07:18:17.914107 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 07:18:17.914154 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 07:18:17.914294 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 07:18:17.914364 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 07:18:17.914399 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0924 07:18:17.914431 (+ 32us) server_negotiation.cc:299] Negotiation successful
0924 07:18:17.914447 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":61,"threads_started":1}
I20260924 07:18:17.915762 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 884
I20260924 07:18:17.920252 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 959
I20260924 07:18:17.924326 10763 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9LiIRP/build/release/bin/kudu with pid 1034
2026-09-24T07:18:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2630 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5381 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (252963 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST